2024-04-26T20:14:59.032 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T20:14:59.033 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_rishabh@teuthology --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone}} --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165 -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165/orig.config.yaml 2024-04-26T20:14:59.084 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:14:59.165 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165 branch: wip-rishabh-testing-20240426.111959 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone}} email: null first_in_suite: false job_id: '7675165' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFJI9/a6xN71scHvRs+a5LoMYoCg+MPzsxjdSAWc0S4CYiqDJV8Blh4sl0X8rm8epzpjduzbDy8WjrR3AzVGZxs= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOiFV00u1Fzyoluyqeb6wqgjcP6WsSXZvL9D3fU5DC4maJJdoBJHMG53mJJygw64fB7I3ioVgzYq8DCD6mjgbgA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T20:14:59.165 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T20:14:59.166 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T20:14:59.166 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T20:14:59.167 INFO:teuthology.task.internal:Checking packages... 2024-04-26T20:14:59.192 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T20:14:59.193 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T20:14:59.193 INFO:teuthology.packaging:ref: None 2024-04-26T20:14:59.193 INFO:teuthology.packaging:tag: None 2024-04-26T20:14:59.193 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T20:14:59.193 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:14:59.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T20:14:59.331 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T20:14:59.333 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T20:14:59.339 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T20:14:59.339 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T20:14:59.432 INFO:teuthology.task.internal:Saving configuration 2024-04-26T20:14:59.449 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T20:14:59.459 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:14:59.478 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:10:06.308296', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFJI9/a6xN71scHvRs+a5LoMYoCg+MPzsxjdSAWc0S4CYiqDJV8Blh4sl0X8rm8epzpjduzbDy8WjrR3AzVGZxs='} 2024-04-26T20:14:59.497 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:10:06.309815', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOiFV00u1Fzyoluyqeb6wqgjcP6WsSXZvL9D3fU5DC4maJJdoBJHMG53mJJygw64fB7I3ioVgzYq8DCD6mjgbgA='} 2024-04-26T20:14:59.497 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T20:14:59.501 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T20:14:59.501 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T20:14:59.501 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T20:14:59.647 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdd4d452160>, signals=[15]) 2024-04-26T20:14:59.647 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T20:14:59.653 INFO:teuthology.task.internal:Opening connections... 2024-04-26T20:14:59.653 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:14:59.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:14:59.735 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:14:59.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:14:59.811 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T20:14:59.816 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-04-26T20:14:59.834 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-04-26T20:14:59.835 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-04-26T20:14:59.892 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-04-26T20:14:59.893 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-04-26T20:14:59.893 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-04-26T20:14:59.893 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:14:59.893 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-04-26T20:14:59.893 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:14:59.893 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:14:59.894 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-04-26T20:14:59.894 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:14:59.894 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:14:59.894 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:14:59.894 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:14:59.894 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:14:59.894 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:14:59.895 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-04-26T20:14:59.912 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-04-26T20:14:59.929 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-04-26T20:14:59.929 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-04-26T20:14:59.986 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-04-26T20:14:59.986 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-04-26T20:14:59.986 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-04-26T20:14:59.986 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:14:59.986 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-04-26T20:14:59.986 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:14:59.986 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:14:59.987 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-04-26T20:14:59.987 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:14:59.987 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:14:59.987 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:14:59.987 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:14:59.987 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:14:59.987 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:14:59.987 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-04-26T20:15:00.003 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T20:15:00.013 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T20:15:00.042 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T20:15:00.043 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:15:00.045 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:15:00.061 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T20:15:00.125 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T20:15:00.126 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:15:00.128 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:15:00.155 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T20:15:00.206 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T20:15:00.221 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T20:15:00.221 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-26T20:15:00.222 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-26T20:15:00.222 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:15:00.222 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:15:00.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:15:00.239 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-04-26T20:15:00.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:15:00.240 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-04-26T20:15:00.295 INFO:teuthology.orchestra.run.smithi134.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:15:00.296 INFO:teuthology.task.kernel:Running kernel on smithi134: 5.14.0-437.el9.x86_64 2024-04-26T20:15:00.296 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum install -y kernel 2024-04-26T20:15:00.298 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:15:00.298 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.14.0-437.el9.x86_64 2024-04-26T20:15:00.298 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum install -y kernel 2024-04-26T20:15:02.952 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.0 MB 00:00 2024-04-26T20:15:03.083 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.0 MB 00:01 2024-04-26T20:15:05.072 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-26T20:15:05.997 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 19 MB 00:01 2024-04-26T20:15:11.306 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.2 MB 00:02 2024-04-26T20:15:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-26T20:15:14.319 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-26T20:15:16.936 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 861 kB/s | 6.2 MB 00:07 2024-04-26T20:15:18.761 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-26T20:15:19.542 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-26T20:15:21.507 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T20:15:21.507 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T20:15:21.563 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 21 MB 00:02 2024-04-26T20:15:21.595 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-26T20:15:21.596 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-04-26T20:15:21.596 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-26T20:15:21.697 DEBUG:teuthology.orchestra.run.smithi037:> echo no | sudo yum reinstall kernel || true 2024-04-26T20:15:22.220 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 08:15:19 PM UTC. 2024-04-26T20:15:22.344 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-26T20:15:22.358 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T20:15:22.358 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-26T20:15:22.359 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T20:15:22.359 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-26T20:15:22.359 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T20:15:22.359 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:15:22.359 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-26T20:15:22.359 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T20:15:22.359 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:15:22.359 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-26T20:15:22.359 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-26T20:15:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:Operation aborted. 2024-04-26T20:15:22.412 INFO:teuthology.orchestra.run.smithi037.stdout:Is this ok [y/N]: 2024-04-26T20:15:22.413 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum reinstall -y kernel || true 2024-04-26T20:15:22.930 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 08:15:19 PM UTC. 2024-04-26T20:15:23.051 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-26T20:15:23.052 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T20:15:23.052 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-26T20:15:23.052 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T20:15:23.052 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-26T20:15:23.052 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T20:15:23.052 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:15:23.052 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-26T20:15:23.052 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T20:15:23.052 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:15:23.053 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-26T20:15:23.053 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-26T20:15:23.053 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-26T20:15:23.344 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-04-26T20:15:23.344 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:15:23.344 INFO:teuthology.orchestra.run.smithi037.stdout:Total 874 kB/s | 255 kB 00:00 2024-04-26T20:15:23.364 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-26T20:15:23.543 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-26T20:15:23.543 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-26T20:15:23.548 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-26T20:15:23.549 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-26T20:15:23.702 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-26T20:15:23.761 INFO:teuthology.orchestra.run.smithi037.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T20:15:23.762 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:15:24.393 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:15:24.393 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T20:15:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:15:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:15:25.118 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalled: 2024-04-26T20:15:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T20:15:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:15:25.119 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-26T20:15:25.282 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T20:15:25.330 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T20:15:25.330 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T20:15:25.331 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T20:15:25.331 DEBUG:teuthology.parallel:result is None 2024-04-26T20:15:26.604 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-26T20:15:28.469 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T20:15:28.469 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T20:15:28.551 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-26T20:15:28.552 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2024-04-26T20:15:28.552 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-04-26T20:15:28.712 DEBUG:teuthology.orchestra.run.smithi134:> echo no | sudo yum reinstall kernel || true 2024-04-26T20:15:29.208 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 08:15:26 PM UTC. 2024-04-26T20:15:29.326 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-26T20:15:29.336 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-26T20:15:29.336 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2024-04-26T20:15:29.336 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-26T20:15:29.336 INFO:teuthology.orchestra.run.smithi134.stdout:Reinstalling: 2024-04-26T20:15:29.336 INFO:teuthology.orchestra.run.smithi134.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T20:15:29.337 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:15:29.337 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-04-26T20:15:29.337 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-26T20:15:29.337 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:15:29.337 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 255 k 2024-04-26T20:15:29.337 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 0 2024-04-26T20:15:29.337 INFO:teuthology.orchestra.run.smithi134.stderr:Operation aborted. 2024-04-26T20:15:29.398 INFO:teuthology.orchestra.run.smithi134.stdout:Is this ok [y/N]: 2024-04-26T20:15:29.398 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum reinstall -y kernel || true 2024-04-26T20:15:29.890 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 08:15:26 PM UTC. 2024-04-26T20:15:30.006 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-26T20:15:30.007 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-26T20:15:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2024-04-26T20:15:30.007 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-26T20:15:30.007 INFO:teuthology.orchestra.run.smithi134.stdout:Reinstalling: 2024-04-26T20:15:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T20:15:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:15:30.007 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-04-26T20:15:30.007 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-26T20:15:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:15:30.008 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 255 k 2024-04-26T20:15:30.008 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 0 2024-04-26T20:15:30.008 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-04-26T20:15:30.173 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.5 MB/s | 255 kB 00:00 2024-04-26T20:15:30.173 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:15:30.173 INFO:teuthology.orchestra.run.smithi134.stdout:Total 1.5 MB/s | 255 kB 00:00 2024-04-26T20:15:30.193 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-04-26T20:15:30.385 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-04-26T20:15:30.385 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-04-26T20:15:30.390 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-04-26T20:15:30.390 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-04-26T20:15:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-04-26T20:15:30.607 INFO:teuthology.orchestra.run.smithi134.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T20:15:30.607 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:15:31.361 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:15:31.361 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T20:15:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:15:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:15:32.077 INFO:teuthology.orchestra.run.smithi134.stdout:Reinstalled: 2024-04-26T20:15:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T20:15:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:15:32.077 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-04-26T20:15:32.267 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T20:15:32.313 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T20:15:32.314 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T20:15:32.315 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T20:15:32.315 DEBUG:teuthology.parallel:result is None 2024-04-26T20:15:32.315 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T20:15:32.321 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T20:15:32.322 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:15:32.324 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:15:32.340 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T20:15:32.364 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T20:15:32.398 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T20:15:32.398 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:15:32.400 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:15:32.432 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T20:15:32.437 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T20:15:32.437 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:15:32.469 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:15:32.507 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:15:32.524 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:15:32.524 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:15:32.542 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:15:32.543 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T20:15:32.550 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T20:15:32.550 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:15:32.553 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:15:32.589 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T20:15:32.597 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T20:15:32.598 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:15:32.621 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:15:32.644 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:15:32.720 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:15:32.767 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:15:32.849 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:15:32.892 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:15:32.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:15:32.958 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:15:33.030 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:15:33.059 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:15:33.141 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:15:33.181 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:15:33.181 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:15:33.244 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-04-26T20:15:33.247 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-04-26T20:15:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:15:33.321 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:15:33.786 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T20:15:33.796 INFO:teuthology.task.internal:Starting timer... 2024-04-26T20:15:33.796 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T20:15:33.818 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T20:15:33.852 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T20:15:33.852 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-04-26T20:15:33.921 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-04-26T20:15:33.923 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-04-26T20:15:33.982 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-04-26T20:15:33.983 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T20:15:33.983 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-04-26T20:15:34.008 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-04-26T20:15:34.009 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-04-26T20:15:34.033 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-04-26T20:15:34.033 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2024-04-26T20:15:34.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:15:34.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:15:34.075 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:15:34.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:15:34.100 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T20:15:34.101 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:15:34.138 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:15:34.168 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T20:15:34.179 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T20:15:34.184 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T20:15:34.311 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T20:15:34.327 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T20:15:34.328 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T20:19:03.449 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-04-26T20:19:03.451 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-26T20:19:03.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:19:03.531 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-04-26T20:19:03.614 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-26T20:19:03.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-26T20:19:03.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:19:03.690 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-04-26T20:19:03.768 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-26T20:19:03.769 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T20:19:03.779 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T20:19:03.779 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T20:19:03.779 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:19:03.782 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T20:19:03.782 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:19:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T20:19:03.825 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T20:19:03.836 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T20:19:03.849 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T20:19:03.849 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-04-26T20:19:03.861 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-04-26T20:19:03.871 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-04-26T20:19:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T20:19:03.881 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-04-26T20:19:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T20:19:03.895 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T20:19:03.908 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T20:19:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:19:03.955 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:19:03.955 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-26T20:19:03.955 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:19:03.955 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:19:03.955 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:19:03.955 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:19:03.962 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:19:03.966 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:19:03.966 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-26T20:19:03.966 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:19:03.966 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:19:03.966 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:19:03.966 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:19:03.967 INFO:teuthology.run_tasks:Running task install... 2024-04-26T20:19:03.975 DEBUG:teuthology.task.install:project ceph 2024-04-26T20:19:03.975 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T20:19:03.975 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T20:19:03.975 INFO:teuthology.task.install:Using flavor: default 2024-04-26T20:19:03.981 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T20:19:03.981 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T20:19:03.982 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T20:19:03.982 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:19:03.985 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T20:19:03.985 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:19:04.200 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T20:19:04.200 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T20:19:04.214 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T20:19:04.214 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T20:19:04.332 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T20:19:04.332 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:19:04.332 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T20:19:04.356 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T20:19:04.357 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T20:19:04.376 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T20:19:04.377 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:19:04.377 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T20:19:04.404 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T20:19:04.404 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T20:19:04.421 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T20:19:04.471 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T20:19:04.495 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T20:19:04.545 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T20:19:04.558 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-04-26T20:19:04.559 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-04-26T20:19:04.610 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-04-26T20:19:04.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-04-26T20:19:04.781 INFO:teuthology.orchestra.run.smithi134.stdout:48 files removed 2024-04-26T20:19:04.808 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T20:19:04.846 INFO:teuthology.orchestra.run.smithi037.stdout:48 files removed 2024-04-26T20:19:04.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T20:19:05.375 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 271 kB/s | 84 kB 00:00 2024-04-26T20:19:05.557 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 203 kB/s | 84 kB 00:00 2024-04-26T20:19:05.833 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-26T20:19:05.883 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-04-26T20:19:06.159 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-04-26T20:19:06.383 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-04-26T20:19:06.859 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-26T20:19:06.942 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-26T20:19:08.661 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-26T20:19:08.938 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-26T20:19:12.778 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-26T20:19:14.061 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.2 MB 00:01 2024-04-26T20:19:14.346 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-26T20:19:15.470 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 16 kB 00:00 2024-04-26T20:19:16.125 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-26T20:19:16.522 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-04-26T20:19:21.380 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 7.1 kB/s | 1.7 kB 00:00 2024-04-26T20:19:21.863 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-26T20:19:23.532 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:19:23.532 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:19:23.543 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T20:19:23.544 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T20:19:23.545 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:19:23.545 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:19:23.547 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:19:23.549 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T20:19:23.550 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:19:23.611 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-26T20:19:23.620 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-26T20:19:23.620 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-04-26T20:19:23.620 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-26T20:19:23.620 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-04-26T20:19:23.620 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T20:19:23.620 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T20:19:23.621 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T20:19:23.622 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-04-26T20:19:23.623 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T20:19:23.624 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:19:23.624 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T20:19:23.624 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T20:19:23.624 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-04-26T20:19:23.624 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T20:19:23.624 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T20:19:23.624 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T20:19:23.624 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T20:19:23.624 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T20:19:23.624 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T20:19:23.624 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T20:19:23.625 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T20:19:23.626 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T20:19:23.627 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T20:19:23.628 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T20:19:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T20:19:23.630 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T20:19:23.630 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T20:19:23.630 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T20:19:23.630 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T20:19:23.630 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T20:19:23.630 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T20:19:23.630 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T20:19:23.630 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T20:19:23.630 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T20:19:23.630 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T20:19:23.630 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T20:19:23.631 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T20:19:23.632 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-04-26T20:19:23.633 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-26T20:19:23.634 INFO:teuthology.orchestra.run.smithi134.stdout:Install 148 Packages 2024-04-26T20:19:23.634 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 4 Packages 2024-04-26T20:19:23.634 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:19:23.634 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 222 M 2024-04-26T20:19:23.634 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-04-26T20:19:24.073 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:19:24.074 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:19:24.085 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T20:19:24.086 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T20:19:24.086 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:19:24.087 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:19:24.089 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:19:24.091 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T20:19:24.092 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:19:24.157 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-26T20:19:24.167 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-26T20:19:24.167 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-04-26T20:19:24.167 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-26T20:19:24.167 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-04-26T20:19:24.167 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T20:19:24.167 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T20:19:24.167 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:19:24.167 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T20:19:24.168 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T20:19:24.169 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-04-26T20:19:24.170 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T20:19:24.171 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:19:24.172 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T20:19:24.173 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T20:19:24.174 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T20:19:24.175 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T20:19:24.176 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T20:19:24.177 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T20:19:24.178 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-04-26T20:19:24.179 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T20:19:24.179 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:19:24.179 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-26T20:19:24.179 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-26T20:19:24.179 INFO:teuthology.orchestra.run.smithi037.stdout:Install 148 Packages 2024-04-26T20:19:24.179 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 4 Packages 2024-04-26T20:19:24.179 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:19:24.179 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 222 M 2024-04-26T20:19:24.179 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-26T20:19:24.898 INFO:teuthology.orchestra.run.smithi134.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-26T20:19:24.915 INFO:teuthology.orchestra.run.smithi037.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-04-26T20:19:25.167 INFO:teuthology.orchestra.run.smithi134.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 3.2 MB/s | 889 kB 00:00 2024-04-26T20:19:25.223 INFO:teuthology.orchestra.run.smithi037.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.8 MB/s | 889 kB 00:00 2024-04-26T20:19:25.234 INFO:teuthology.orchestra.run.smithi134.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.1 MB/s | 145 kB 00:00 2024-04-26T20:19:25.282 INFO:teuthology.orchestra.run.smithi037.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.4 MB/s | 145 kB 00:00 2024-04-26T20:19:25.609 INFO:teuthology.orchestra.run.smithi134.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 5.4 MB/s | 5.4 MB 00:00 2024-04-26T20:19:25.649 INFO:teuthology.orchestra.run.smithi037.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 5.9 MB/s | 5.4 MB 00:00 2024-04-26T20:19:25.659 INFO:teuthology.orchestra.run.smithi134.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 5.6 MB/s | 2.4 MB 00:00 2024-04-26T20:19:25.826 INFO:teuthology.orchestra.run.smithi134.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 7.0 MB/s | 1.5 MB 00:00 2024-04-26T20:19:25.849 INFO:teuthology.orchestra.run.smithi037.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 4.2 MB/s | 2.4 MB 00:00 2024-04-26T20:19:25.883 INFO:teuthology.orchestra.run.smithi037.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 6.5 MB/s | 1.5 MB 00:00 2024-04-26T20:19:26.493 INFO:teuthology.orchestra.run.smithi134.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 5.6 MB/s | 4.7 MB 00:00 2024-04-26T20:19:27.075 INFO:teuthology.orchestra.run.smithi037.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 3.8 MB/s | 4.7 MB 00:01 2024-04-26T20:19:28.235 INFO:teuthology.orchestra.run.smithi134.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 7.0 MB/s | 17 MB 00:02 2024-04-26T20:19:28.291 INFO:teuthology.orchestra.run.smithi037.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 7.0 MB/s | 17 MB 00:02 2024-04-26T20:19:28.333 INFO:teuthology.orchestra.run.smithi037.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 607 kB/s | 25 kB 00:00 2024-04-26T20:19:28.360 INFO:teuthology.orchestra.run.smithi134.stdout:(9/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 5.7 MB/s | 11 MB 00:01 2024-04-26T20:19:28.410 INFO:teuthology.orchestra.run.smithi134.stdout:(10/152): ceph-selinux-19.0.0-3306.gbc612ffb.el 144 kB/s | 25 kB 00:00 2024-04-26T20:19:28.502 INFO:teuthology.orchestra.run.smithi134.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.4 MB/s | 225 kB 00:00 2024-04-26T20:19:28.586 INFO:teuthology.orchestra.run.smithi134.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 324 kB/s | 27 kB 00:00 2024-04-26T20:19:28.669 INFO:teuthology.orchestra.run.smithi134.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 381 kB/s | 32 kB 00:00 2024-04-26T20:19:28.845 INFO:teuthology.orchestra.run.smithi134.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.2 MB/s | 747 kB 00:00 2024-04-26T20:19:28.937 INFO:teuthology.orchestra.run.smithi134.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.7 MB/s | 163 kB 00:00 2024-04-26T20:19:29.029 INFO:teuthology.orchestra.run.smithi134.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.4 MB/s | 127 kB 00:00 2024-04-26T20:19:29.196 INFO:teuthology.orchestra.run.smithi134.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 2.9 MB/s | 501 kB 00:00 2024-04-26T20:19:29.267 INFO:teuthology.orchestra.run.smithi037.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 4.8 MB/s | 11 MB 00:02 2024-04-26T20:19:29.368 INFO:teuthology.orchestra.run.smithi037.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.2 MB/s | 225 kB 00:00 2024-04-26T20:19:29.459 INFO:teuthology.orchestra.run.smithi037.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 295 kB/s | 27 kB 00:00 2024-04-26T20:19:29.551 INFO:teuthology.orchestra.run.smithi037.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 346 kB/s | 32 kB 00:00 2024-04-26T20:19:29.818 INFO:teuthology.orchestra.run.smithi037.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 2.7 MB/s | 747 kB 00:00 2024-04-26T20:19:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 163 kB 00:00 2024-04-26T20:19:29.961 INFO:teuthology.orchestra.run.smithi037.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 2.1 MB/s | 127 kB 00:00 2024-04-26T20:19:29.963 INFO:teuthology.orchestra.run.smithi134.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 7.0 MB/s | 5.4 MB 00:00 2024-04-26T20:19:30.038 INFO:teuthology.orchestra.run.smithi134.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 603 kB/s | 45 kB 00:00 2024-04-26T20:19:30.094 INFO:teuthology.orchestra.run.smithi037.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 3.7 MB/s | 501 kB 00:00 2024-04-26T20:19:30.105 INFO:teuthology.orchestra.run.smithi134.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 2.0 MB/s | 135 kB 00:00 2024-04-26T20:19:30.180 INFO:teuthology.orchestra.run.smithi134.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.1 MB/s | 161 kB 00:00 2024-04-26T20:19:30.272 INFO:teuthology.orchestra.run.smithi134.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.4 MB/s | 321 kB 00:00 2024-04-26T20:19:30.356 INFO:teuthology.orchestra.run.smithi134.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.5 MB/s | 298 kB 00:00 2024-04-26T20:19:30.431 INFO:teuthology.orchestra.run.smithi134.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.3 MB/s | 99 kB 00:00 2024-04-26T20:19:30.506 INFO:teuthology.orchestra.run.smithi134.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-26T20:19:31.078 INFO:teuthology.orchestra.run.smithi037.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 5.5 MB/s | 5.4 MB 00:00 2024-04-26T20:19:31.082 INFO:teuthology.orchestra.run.smithi134.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 5.4 MB/s | 3.1 MB 00:00 2024-04-26T20:19:31.128 INFO:teuthology.orchestra.run.smithi037.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 908 kB/s | 45 kB 00:00 2024-04-26T20:19:31.149 INFO:teuthology.orchestra.run.smithi134.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-26T20:19:31.195 INFO:teuthology.orchestra.run.smithi037.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 2.0 MB/s | 135 kB 00:00 2024-04-26T20:19:31.224 INFO:teuthology.orchestra.run.smithi134.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 458 kB/s | 34 kB 00:00 2024-04-26T20:19:31.262 INFO:teuthology.orchestra.run.smithi037.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.4 MB/s | 161 kB 00:00 2024-04-26T20:19:31.300 INFO:teuthology.orchestra.run.smithi134.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.8 MB/s | 142 kB 00:00 2024-04-26T20:19:31.346 INFO:teuthology.orchestra.run.smithi037.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 321 kB 00:00 2024-04-26T20:19:31.437 INFO:teuthology.orchestra.run.smithi037.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.2 MB/s | 298 kB 00:00 2024-04-26T20:19:31.521 INFO:teuthology.orchestra.run.smithi037.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T20:19:31.605 INFO:teuthology.orchestra.run.smithi037.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-26T20:19:31.842 INFO:teuthology.orchestra.run.smithi134.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 6.5 MB/s | 3.5 MB 00:00 2024-04-26T20:19:32.259 INFO:teuthology.orchestra.run.smithi037.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 4.8 MB/s | 3.1 MB 00:00 2024-04-26T20:19:32.347 INFO:teuthology.orchestra.run.smithi037.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-26T20:19:32.430 INFO:teuthology.orchestra.run.smithi037.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 411 kB/s | 34 kB 00:00 2024-04-26T20:19:32.522 INFO:teuthology.orchestra.run.smithi037.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.5 MB/s | 142 kB 00:00 2024-04-26T20:19:33.173 INFO:teuthology.orchestra.run.smithi037.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 5.4 MB/s | 3.5 MB 00:00 2024-04-26T20:19:33.242 INFO:teuthology.orchestra.run.smithi134.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-04-26T20:19:33.492 INFO:teuthology.orchestra.run.smithi134.stdout:(32/152): ceph-common-19.0.0-3306.gbc612ffb.el9 2.4 MB/s | 21 MB 00:08 2024-04-26T20:19:33.526 INFO:teuthology.orchestra.run.smithi134.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 878 kB/s | 249 kB 00:00 2024-04-26T20:19:33.584 INFO:teuthology.orchestra.run.smithi134.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 537 kB/s | 49 kB 00:00 2024-04-26T20:19:33.618 INFO:teuthology.orchestra.run.smithi134.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 179 kB/s | 16 kB 00:00 2024-04-26T20:19:33.760 INFO:teuthology.orchestra.run.smithi134.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 273 kB 00:00 2024-04-26T20:19:33.793 INFO:teuthology.orchestra.run.smithi134.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 4.3 MB/s | 762 kB 00:00 2024-04-26T20:19:33.835 INFO:teuthology.orchestra.run.smithi134.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 242 kB/s | 18 kB 00:00 2024-04-26T20:19:33.917 INFO:teuthology.orchestra.run.smithi037.stdout:(31/152): ceph-common-19.0.0-3306.gbc612ffb.el9 2.3 MB/s | 21 MB 00:09 2024-04-26T20:19:33.952 INFO:teuthology.orchestra.run.smithi134.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 507 kB/s | 80 kB 00:00 2024-04-26T20:19:34.036 INFO:teuthology.orchestra.run.smithi134.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-26T20:19:34.169 INFO:teuthology.orchestra.run.smithi134.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-04-26T20:19:34.203 INFO:teuthology.orchestra.run.smithi134.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-04-26T20:19:34.253 INFO:teuthology.orchestra.run.smithi134.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-26T20:19:34.304 INFO:teuthology.orchestra.run.smithi134.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-26T20:19:34.359 INFO:teuthology.orchestra.run.smithi037.stdout:(32/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 8.2 MB/s | 49 MB 00:06 2024-04-26T20:19:34.362 INFO:teuthology.orchestra.run.smithi134.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-26T20:19:34.384 INFO:teuthology.orchestra.run.smithi037.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 533 kB/s | 249 kB 00:00 2024-04-26T20:19:34.409 INFO:teuthology.orchestra.run.smithi037.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 988 kB/s | 49 kB 00:00 2024-04-26T20:19:34.454 INFO:teuthology.orchestra.run.smithi134.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-26T20:19:34.488 INFO:teuthology.orchestra.run.smithi134.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-04-26T20:19:34.502 INFO:teuthology.orchestra.run.smithi037.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 5.6 MB/s | 7.4 MB 00:01 2024-04-26T20:19:34.526 INFO:teuthology.orchestra.run.smithi037.stdout:(36/152): ceph-prometheus-alerts-19.0.0-3306.gb 116 kB/s | 16 kB 00:00 2024-04-26T20:19:34.551 INFO:teuthology.orchestra.run.smithi037.stdout:(37/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.9 MB/s | 273 kB 00:00 2024-04-26T20:19:34.594 INFO:teuthology.orchestra.run.smithi037.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 273 kB/s | 18 kB 00:00 2024-04-26T20:19:34.618 INFO:teuthology.orchestra.run.smithi037.stdout:(39/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 6.4 MB/s | 762 kB 00:00 2024-04-26T20:19:34.663 INFO:teuthology.orchestra.run.smithi134.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-04-26T20:19:35.169 INFO:teuthology.orchestra.run.smithi037.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 130 kB/s | 80 kB 00:00 2024-04-26T20:19:35.172 INFO:teuthology.orchestra.run.smithi134.stdout:(49/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 7.2 MB/s | 49 MB 00:06 2024-04-26T20:19:35.197 INFO:teuthology.orchestra.run.smithi134.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 200 kB/s | 107 kB 00:00 2024-04-26T20:19:35.239 INFO:teuthology.orchestra.run.smithi134.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-04-26T20:19:35.264 INFO:teuthology.orchestra.run.smithi134.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 234 kB/s | 21 kB 00:00 2024-04-26T20:19:35.290 INFO:teuthology.orchestra.run.smithi134.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-04-26T20:19:35.332 INFO:teuthology.orchestra.run.smithi134.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 355 kB/s | 33 kB 00:00 2024-04-26T20:19:35.386 INFO:teuthology.orchestra.run.smithi037.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 255 kB/s | 195 kB 00:00 2024-04-26T20:19:35.390 INFO:teuthology.orchestra.run.smithi134.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 181 kB/s | 18 kB 00:00 2024-04-26T20:19:35.415 INFO:teuthology.orchestra.run.smithi134.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-04-26T20:19:35.475 INFO:teuthology.orchestra.run.smithi134.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-26T20:19:35.499 INFO:teuthology.orchestra.run.smithi134.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 539 kB/s | 45 kB 00:00 2024-04-26T20:19:35.658 INFO:teuthology.orchestra.run.smithi134.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-04-26T20:19:35.750 INFO:teuthology.orchestra.run.smithi134.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 2.3 MB/s | 582 kB 00:00 2024-04-26T20:19:35.825 INFO:teuthology.orchestra.run.smithi134.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 152 kB/s | 25 kB 00:00 2024-04-26T20:19:35.884 INFO:teuthology.orchestra.run.smithi134.stdout:(62/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.8 MB/s | 3.0 MB 00:00 2024-04-26T20:19:35.909 INFO:teuthology.orchestra.run.smithi134.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 298 kB/s | 47 kB 00:00 2024-04-26T20:19:35.934 INFO:teuthology.orchestra.run.smithi134.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-26T20:19:35.968 INFO:teuthology.orchestra.run.smithi134.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-26T20:19:35.986 INFO:teuthology.orchestra.run.smithi037.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 577 kB/s | 803 kB 00:01 2024-04-26T20:19:35.994 INFO:teuthology.orchestra.run.smithi134.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 420 kB/s | 35 kB 00:00 2024-04-26T20:19:36.028 INFO:teuthology.orchestra.run.smithi037.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 295 kB/s | 253 kB 00:00 2024-04-26T20:19:36.119 INFO:teuthology.orchestra.run.smithi134.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-26T20:19:36.211 INFO:teuthology.orchestra.run.smithi134.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-04-26T20:19:36.278 INFO:teuthology.orchestra.run.smithi134.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-26T20:19:36.303 INFO:teuthology.orchestra.run.smithi134.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-26T20:19:36.345 INFO:teuthology.orchestra.run.smithi134.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 3.9 MB/s | 911 kB 00:00 2024-04-26T20:19:36.728 INFO:teuthology.orchestra.run.smithi037.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 144 kB/s | 106 kB 00:00 2024-04-26T20:19:36.795 INFO:teuthology.orchestra.run.smithi037.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 176 kB/s | 135 kB 00:00 2024-04-26T20:19:36.853 INFO:teuthology.orchestra.run.smithi134.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 172 kB/s | 87 kB 00:00 2024-04-26T20:19:36.995 INFO:teuthology.orchestra.run.smithi134.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 1.5 MB/s | 1.0 MB 00:00 2024-04-26T20:19:37.346 INFO:teuthology.orchestra.run.smithi134.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 697 kB/s | 244 kB 00:00 2024-04-26T20:19:37.471 INFO:teuthology.orchestra.run.smithi134.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-04-26T20:19:37.495 INFO:teuthology.orchestra.run.smithi037.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 603 kB/s | 1.2 MB 00:02 2024-04-26T20:19:37.521 INFO:teuthology.orchestra.run.smithi037.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 158 kB/s | 125 kB 00:00 2024-04-26T20:19:37.579 INFO:teuthology.orchestra.run.smithi134.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 4.4 MB/s | 5.8 MB 00:01 2024-04-26T20:19:37.638 INFO:teuthology.orchestra.run.smithi134.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 153 kB/s | 26 kB 00:00 2024-04-26T20:19:37.671 INFO:teuthology.orchestra.run.smithi037.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 245 kB/s | 215 kB 00:00 2024-04-26T20:19:37.680 INFO:teuthology.orchestra.run.smithi134.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-04-26T20:19:37.705 INFO:teuthology.orchestra.run.smithi134.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-26T20:19:37.721 INFO:teuthology.orchestra.run.smithi037.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-04-26T20:19:37.731 INFO:teuthology.orchestra.run.smithi134.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-04-26T20:19:37.763 INFO:teuthology.orchestra.run.smithi037.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-04-26T20:19:37.788 INFO:teuthology.orchestra.run.smithi037.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 323 kB/s | 21 kB 00:00 2024-04-26T20:19:37.806 INFO:teuthology.orchestra.run.smithi134.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.3 MB/s | 6.0 MB 00:00 2024-04-26T20:19:37.814 INFO:teuthology.orchestra.run.smithi037.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-26T20:19:37.831 INFO:teuthology.orchestra.run.smithi134.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-26T20:19:37.839 INFO:teuthology.orchestra.run.smithi037.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-04-26T20:19:37.864 INFO:teuthology.orchestra.run.smithi037.stdout:(54/152): boost-program-options-1.75.0-8.el9.x8 554 kB/s | 107 kB 00:00 2024-04-26T20:19:37.889 INFO:teuthology.orchestra.run.smithi037.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-04-26T20:19:37.890 INFO:teuthology.orchestra.run.smithi134.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-26T20:19:37.914 INFO:teuthology.orchestra.run.smithi037.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-26T20:19:37.915 INFO:teuthology.orchestra.run.smithi134.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 924 kB/s | 77 kB 00:00 2024-04-26T20:19:37.940 INFO:teuthology.orchestra.run.smithi037.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-26T20:19:37.940 INFO:teuthology.orchestra.run.smithi134.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-04-26T20:19:37.965 INFO:teuthology.orchestra.run.smithi037.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-04-26T20:19:37.974 INFO:teuthology.orchestra.run.smithi134.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-26T20:19:38.024 INFO:teuthology.orchestra.run.smithi037.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-26T20:19:38.041 INFO:teuthology.orchestra.run.smithi134.stdout:(87/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-26T20:19:38.066 INFO:teuthology.orchestra.run.smithi134.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-26T20:19:38.074 INFO:teuthology.orchestra.run.smithi037.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-26T20:19:38.091 INFO:teuthology.orchestra.run.smithi037.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 379 kB/s | 25 kB 00:00 2024-04-26T20:19:38.092 INFO:teuthology.orchestra.run.smithi134.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T20:19:38.116 INFO:teuthology.orchestra.run.smithi037.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-26T20:19:38.117 INFO:teuthology.orchestra.run.smithi134.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-04-26T20:19:38.133 INFO:teuthology.orchestra.run.smithi037.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 805 kB/s | 47 kB 00:00 2024-04-26T20:19:38.142 INFO:teuthology.orchestra.run.smithi134.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-04-26T20:19:38.159 INFO:teuthology.orchestra.run.smithi037.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-26T20:19:38.184 INFO:teuthology.orchestra.run.smithi037.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-04-26T20:19:38.209 INFO:teuthology.orchestra.run.smithi037.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-26T20:19:38.318 INFO:teuthology.orchestra.run.smithi134.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 208 kB/s | 42 kB 00:00 2024-04-26T20:19:38.359 INFO:teuthology.orchestra.run.smithi134.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 204 kB/s | 44 kB 00:00 2024-04-26T20:19:38.443 INFO:teuthology.orchestra.run.smithi037.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-04-26T20:19:38.485 INFO:teuthology.orchestra.run.smithi134.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.3 MB/s | 221 kB 00:00 2024-04-26T20:19:38.510 INFO:teuthology.orchestra.run.smithi037.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-26T20:19:38.510 INFO:teuthology.orchestra.run.smithi134.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 239 kB/s | 36 kB 00:00 2024-04-26T20:19:38.552 INFO:teuthology.orchestra.run.smithi037.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-04-26T20:19:38.577 INFO:teuthology.orchestra.run.smithi037.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-26T20:19:38.611 INFO:teuthology.orchestra.run.smithi037.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 911 kB 00:00 2024-04-26T20:19:38.627 INFO:teuthology.orchestra.run.smithi134.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 334 kB/s | 47 kB 00:00 2024-04-26T20:19:38.670 INFO:teuthology.orchestra.run.smithi037.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-26T20:19:38.694 INFO:teuthology.orchestra.run.smithi037.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-04-26T20:19:38.710 INFO:teuthology.orchestra.run.smithi134.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-04-26T20:19:38.744 INFO:teuthology.orchestra.run.smithi134.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 494 kB/s | 58 kB 00:00 2024-04-26T20:19:38.753 INFO:teuthology.orchestra.run.smithi037.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-26T20:19:38.794 INFO:teuthology.orchestra.run.smithi134.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-26T20:19:38.820 INFO:teuthology.orchestra.run.smithi134.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-04-26T20:19:38.820 INFO:teuthology.orchestra.run.smithi037.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-26T20:19:38.878 INFO:teuthology.orchestra.run.smithi037.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-04-26T20:19:38.929 INFO:teuthology.orchestra.run.smithi037.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-26T20:19:38.995 INFO:teuthology.orchestra.run.smithi134.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 224 kB/s | 45 kB 00:00 2024-04-26T20:19:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-26T20:19:39.096 INFO:teuthology.orchestra.run.smithi037.stdout:(79/152): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-04-26T20:19:39.122 INFO:teuthology.orchestra.run.smithi037.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 297 kB/s | 35 kB 00:00 2024-04-26T20:19:39.129 INFO:teuthology.orchestra.run.smithi134.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-04-26T20:19:39.197 INFO:teuthology.orchestra.run.smithi037.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-26T20:19:39.255 INFO:teuthology.orchestra.run.smithi037.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-26T20:19:39.329 INFO:teuthology.orchestra.run.smithi134.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-04-26T20:19:39.339 INFO:teuthology.orchestra.run.smithi037.stdout:(83/152): postgresql-server-13.14-1.el9.x86_64. 7.3 MB/s | 5.8 MB 00:00 2024-04-26T20:19:39.364 INFO:teuthology.orchestra.run.smithi037.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 710 kB/s | 77 kB 00:00 2024-04-26T20:19:39.431 INFO:teuthology.orchestra.run.smithi037.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-26T20:19:39.456 INFO:teuthology.orchestra.run.smithi037.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 360 kB/s | 42 kB 00:00 2024-04-26T20:19:39.481 INFO:teuthology.orchestra.run.smithi037.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-04-26T20:19:39.507 INFO:teuthology.orchestra.run.smithi037.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 159 kB 00:00 2024-04-26T20:19:39.540 INFO:teuthology.orchestra.run.smithi037.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 279 kB 00:00 2024-04-26T20:19:39.565 INFO:teuthology.orchestra.run.smithi037.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-04-26T20:19:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 641 kB/s | 54 kB 00:00 2024-04-26T20:19:39.638 INFO:teuthology.orchestra.run.smithi134.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-04-26T20:19:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-04-26T20:19:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 586 kB/s | 44 kB 00:00 2024-04-26T20:19:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-26T20:19:39.725 INFO:teuthology.orchestra.run.smithi037.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 611 kB/s | 36 kB 00:00 2024-04-26T20:19:39.751 INFO:teuthology.orchestra.run.smithi037.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-26T20:19:39.776 INFO:teuthology.orchestra.run.smithi037.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-26T20:19:39.801 INFO:teuthology.orchestra.run.smithi037.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T20:19:39.813 INFO:teuthology.orchestra.run.smithi134.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 678 kB/s | 309 kB 00:00 2024-04-26T20:19:39.827 INFO:teuthology.orchestra.run.smithi037.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-26T20:19:39.838 INFO:teuthology.orchestra.run.smithi134.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 97 kB/s | 19 kB 00:00 2024-04-26T20:19:39.852 INFO:teuthology.orchestra.run.smithi037.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T20:19:39.863 INFO:teuthology.orchestra.run.smithi134.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 747 kB/s | 548 kB 00:00 2024-04-26T20:19:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-04-26T20:19:39.961 INFO:teuthology.orchestra.run.smithi037.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-04-26T20:19:40.031 INFO:teuthology.orchestra.run.smithi134.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 135 kB/s | 26 kB 00:00 2024-04-26T20:19:40.055 INFO:teuthology.orchestra.run.smithi134.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 253 kB/s | 49 kB 00:00 2024-04-26T20:19:40.122 INFO:teuthology.orchestra.run.smithi134.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-26T20:19:40.148 INFO:teuthology.orchestra.run.smithi134.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2024-04-26T20:19:40.173 INFO:teuthology.orchestra.run.smithi134.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-26T20:19:40.215 INFO:teuthology.orchestra.run.smithi134.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-04-26T20:19:40.248 INFO:teuthology.orchestra.run.smithi134.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 7.2 MB/s | 548 kB 00:00 2024-04-26T20:19:40.274 INFO:teuthology.orchestra.run.smithi134.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 436 kB/s | 43 kB 00:00 2024-04-26T20:19:40.299 INFO:teuthology.orchestra.run.smithi134.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 382 kB/s | 32 kB 00:00 2024-04-26T20:19:40.319 INFO:teuthology.orchestra.run.smithi037.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-26T20:19:40.325 INFO:teuthology.orchestra.run.smithi134.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 198 kB/s | 15 kB 00:00 2024-04-26T20:19:40.344 INFO:teuthology.orchestra.run.smithi037.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-04-26T20:19:40.366 INFO:teuthology.orchestra.run.smithi134.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-26T20:19:40.400 INFO:teuthology.orchestra.run.smithi134.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-26T20:19:40.425 INFO:teuthology.orchestra.run.smithi134.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-04-26T20:19:40.451 INFO:teuthology.orchestra.run.smithi134.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 425 kB/s | 35 kB 00:00 2024-04-26T20:19:40.561 INFO:teuthology.orchestra.run.smithi134.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-04-26T20:19:40.627 INFO:teuthology.orchestra.run.smithi134.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-04-26T20:19:40.652 INFO:teuthology.orchestra.run.smithi134.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 722 kB/s | 144 kB 00:00 2024-04-26T20:19:40.694 INFO:teuthology.orchestra.run.smithi134.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 82 kB/s | 11 kB 00:00 2024-04-26T20:19:40.760 INFO:teuthology.orchestra.run.smithi134.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-26T20:19:40.786 INFO:teuthology.orchestra.run.smithi134.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 173 kB/s | 23 kB 00:00 2024-04-26T20:19:40.811 INFO:teuthology.orchestra.run.smithi134.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-04-26T20:19:40.844 INFO:teuthology.orchestra.run.smithi134.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-04-26T20:19:40.895 INFO:teuthology.orchestra.run.smithi037.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 35 kB/s | 19 kB 00:00 2024-04-26T20:19:40.903 INFO:teuthology.orchestra.run.smithi134.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-26T20:19:40.928 INFO:teuthology.orchestra.run.smithi134.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-04-26T20:19:40.937 INFO:teuthology.orchestra.run.smithi037.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 562 kB/s | 548 kB 00:00 2024-04-26T20:19:40.954 INFO:teuthology.orchestra.run.smithi134.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 724 kB/s | 79 kB 00:00 2024-04-26T20:19:40.987 INFO:teuthology.orchestra.run.smithi134.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 692 kB/s | 58 kB 00:00 2024-04-26T20:19:41.037 INFO:teuthology.orchestra.run.smithi134.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-26T20:19:41.062 INFO:teuthology.orchestra.run.smithi134.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-26T20:19:41.088 INFO:teuthology.orchestra.run.smithi134.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-04-26T20:19:41.113 INFO:teuthology.orchestra.run.smithi134.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-26T20:19:41.137 INFO:teuthology.orchestra.run.smithi037.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 378 kB/s | 309 kB 00:00 2024-04-26T20:19:41.138 INFO:teuthology.orchestra.run.smithi134.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-26T20:19:41.172 INFO:teuthology.orchestra.run.smithi134.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-26T20:19:41.198 INFO:teuthology.orchestra.run.smithi134.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 705 kB/s | 59 kB 00:00 2024-04-26T20:19:41.223 INFO:teuthology.orchestra.run.smithi134.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-04-26T20:19:41.265 INFO:teuthology.orchestra.run.smithi134.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-04-26T20:19:41.290 INFO:teuthology.orchestra.run.smithi134.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-26T20:19:41.323 INFO:teuthology.orchestra.run.smithi134.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 893 kB/s | 90 kB 00:00 2024-04-26T20:19:41.374 INFO:teuthology.orchestra.run.smithi134.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-26T20:19:41.399 INFO:teuthology.orchestra.run.smithi134.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-26T20:19:41.425 INFO:teuthology.orchestra.run.smithi134.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-26T20:19:41.462 INFO:teuthology.orchestra.run.smithi037.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 49 kB/s | 26 kB 00:00 2024-04-26T20:19:41.517 INFO:teuthology.orchestra.run.smithi134.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T20:19:41.604 INFO:teuthology.orchestra.run.smithi037.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 104 kB/s | 49 kB 00:00 2024-04-26T20:19:41.817 INFO:teuthology.orchestra.run.smithi134.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.5 MB/s | 1.3 MB 00:00 2024-04-26T20:19:42.017 INFO:teuthology.orchestra.run.smithi134.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 5.5 MB/s | 3.4 MB 00:00 2024-04-26T20:19:42.104 INFO:teuthology.orchestra.run.smithi037.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 105 kB/s | 67 kB 00:00 2024-04-26T20:19:42.125 INFO:teuthology.orchestra.run.smithi134.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.8 MB/s | 2.4 MB 00:00 2024-04-26T20:19:42.580 INFO:teuthology.orchestra.run.smithi037.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 155 kB/s | 151 kB 00:00 2024-04-26T20:19:42.634 INFO:teuthology.orchestra.run.smithi134.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 2.6 MB/s | 3.1 MB 00:01 2024-04-26T20:19:42.642 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:19:42.642 INFO:teuthology.orchestra.run.smithi134.stdout:Total 12 MB/s | 222 MB 00:19 2024-04-26T20:19:43.022 INFO:teuthology.orchestra.run.smithi037.stdout:(112/152): libarrow-9.0.0-11.el9.x86_64.rpm 2.1 MB/s | 4.4 MB 00:02 2024-04-26T20:19:43.080 INFO:teuthology.orchestra.run.smithi037.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 860 kB/s | 839 kB 00:00 2024-04-26T20:19:43.139 INFO:teuthology.orchestra.run.smithi037.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 983 kB/s | 548 kB 00:00 2024-04-26T20:19:43.164 INFO:teuthology.orchestra.run.smithi037.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 307 kB/s | 43 kB 00:00 2024-04-26T20:19:43.189 INFO:teuthology.orchestra.run.smithi037.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-04-26T20:19:43.214 INFO:teuthology.orchestra.run.smithi037.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-26T20:19:43.281 INFO:teuthology.orchestra.run.smithi037.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-26T20:19:43.332 INFO:teuthology.orchestra.run.smithi037.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-04-26T20:19:43.365 INFO:teuthology.orchestra.run.smithi037.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-04-26T20:19:43.390 INFO:teuthology.orchestra.run.smithi037.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-04-26T20:19:43.424 INFO:teuthology.orchestra.run.smithi037.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-26T20:19:43.675 INFO:teuthology.orchestra.run.smithi037.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-04-26T20:19:43.700 INFO:teuthology.orchestra.run.smithi037.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 468 kB/s | 144 kB 00:00 2024-04-26T20:19:43.717 INFO:teuthology.orchestra.run.smithi037.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 37 kB/s | 11 kB 00:00 2024-04-26T20:19:43.719 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-04-26T20:19:43.742 INFO:teuthology.orchestra.run.smithi037.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-04-26T20:19:43.792 INFO:teuthology.orchestra.run.smithi037.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-04-26T20:19:43.809 INFO:teuthology.orchestra.run.smithi037.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-26T20:19:43.826 INFO:teuthology.orchestra.run.smithi037.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-04-26T20:19:43.864 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-04-26T20:19:43.864 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-04-26T20:19:43.960 INFO:teuthology.orchestra.run.smithi037.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-26T20:19:43.977 INFO:teuthology.orchestra.run.smithi037.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 309 kB/s | 46 kB 00:00 2024-04-26T20:19:44.002 INFO:teuthology.orchestra.run.smithi037.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 450 kB/s | 79 kB 00:00 2024-04-26T20:19:44.027 INFO:teuthology.orchestra.run.smithi037.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 867 kB/s | 58 kB 00:00 2024-04-26T20:19:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-26T20:19:44.119 INFO:teuthology.orchestra.run.smithi037.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-26T20:19:44.144 INFO:teuthology.orchestra.run.smithi037.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 768 kB/s | 90 kB 00:00 2024-04-26T20:19:44.161 INFO:teuthology.orchestra.run.smithi037.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-04-26T20:19:44.220 INFO:teuthology.orchestra.run.smithi037.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 410 kB/s | 31 kB 00:00 2024-04-26T20:19:44.245 INFO:teuthology.orchestra.run.smithi037.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-26T20:19:44.270 INFO:teuthology.orchestra.run.smithi037.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-04-26T20:19:44.296 INFO:teuthology.orchestra.run.smithi037.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-04-26T20:19:44.346 INFO:teuthology.orchestra.run.smithi037.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 681 kB/s | 51 kB 00:00 2024-04-26T20:19:44.371 INFO:teuthology.orchestra.run.smithi037.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-26T20:19:44.396 INFO:teuthology.orchestra.run.smithi037.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-04-26T20:19:44.447 INFO:teuthology.orchestra.run.smithi037.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-26T20:19:44.472 INFO:teuthology.orchestra.run.smithi037.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-26T20:19:44.497 INFO:teuthology.orchestra.run.smithi037.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-26T20:19:44.648 INFO:teuthology.orchestra.run.smithi037.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-26T20:19:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 5.5 MB/s | 3.1 MB 00:00 2024-04-26T20:19:45.115 INFO:teuthology.orchestra.run.smithi037.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 5.3 MB/s | 3.4 MB 00:00 2024-04-26T20:19:45.628 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-04-26T20:19:45.629 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-04-26T20:19:46.690 INFO:teuthology.orchestra.run.smithi037.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 677 kB/s | 1.3 MB 00:02 2024-04-26T20:19:48.231 INFO:teuthology.orchestra.run.smithi037.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 774 kB/s | 2.4 MB 00:03 2024-04-26T20:19:48.237 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:19:48.237 INFO:teuthology.orchestra.run.smithi037.stdout:Total 9.2 MB/s | 222 MB 00:24 2024-04-26T20:19:49.347 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-26T20:19:49.508 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-26T20:19:49.508 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-26T20:19:51.392 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-26T20:19:51.393 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-26T20:19:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-04-26T20:19:51.821 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T20:19:51.859 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T20:19:51.921 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T20:19:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T20:19:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T20:19:52.031 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T20:19:52.360 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T20:19:52.473 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:19:52.595 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:19:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:19:52.708 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:19:52.745 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:19:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T20:19:52.942 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T20:19:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:19:53.090 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:19:53.113 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:19:53.175 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:19:53.197 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:19:53.221 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:19:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:19:53.413 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:19:53.523 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T20:19:53.619 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T20:19:53.663 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T20:19:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T20:19:53.772 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T20:19:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T20:19:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T20:19:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T20:19:53.926 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T20:19:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T20:19:54.049 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T20:19:54.079 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T20:19:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T20:19:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T20:19:54.242 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T20:19:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T20:19:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T20:19:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T20:19:54.429 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T20:19:54.503 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T20:19:54.638 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T20:19:54.689 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T20:19:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T20:19:55.302 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T20:19:55.331 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T20:19:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T20:19:55.412 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T20:19:55.462 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T20:19:55.640 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T20:19:55.698 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T20:19:55.752 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T20:19:55.818 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T20:19:55.848 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T20:19:55.885 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T20:19:55.932 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T20:19:55.967 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T20:19:56.011 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T20:19:56.078 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T20:19:56.214 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T20:19:56.258 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T20:19:56.289 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T20:19:56.330 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T20:19:56.426 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T20:19:56.455 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T20:19:56.495 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T20:19:56.521 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T20:19:56.546 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T20:19:56.629 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T20:19:56.764 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T20:19:56.803 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T20:19:56.831 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T20:19:56.871 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T20:19:56.913 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T20:19:56.969 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T20:19:56.999 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T20:19:57.034 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T20:19:57.067 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T20:19:57.109 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T20:19:57.143 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T20:19:57.238 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T20:19:57.290 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T20:19:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T20:19:57.369 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T20:19:57.468 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T20:19:57.514 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T20:19:57.633 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-26T20:19:57.719 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T20:19:57.766 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T20:19:57.821 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T20:19:57.870 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T20:19:57.895 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T20:19:57.939 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T20:19:58.238 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T20:19:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T20:19:58.286 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T20:19:58.321 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T20:19:58.396 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:19:58.420 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T20:19:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T20:19:58.521 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:19:58.547 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:19:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T20:19:58.598 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T20:19:58.640 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:19:58.668 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:19:58.783 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T20:19:58.821 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T20:19:58.825 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T20:19:58.874 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T20:19:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T20:19:58.901 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:19:59.007 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:19:59.036 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:19:59.082 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:19:59.104 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:19:59.126 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:19:59.321 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:19:59.353 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:19:59.464 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T20:19:59.551 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T20:19:59.561 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T20:19:59.603 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T20:19:59.639 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T20:19:59.714 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T20:19:59.761 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T20:19:59.762 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T20:19:59.806 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T20:19:59.847 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T20:19:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T20:19:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T20:20:00.007 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T20:20:00.035 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T20:20:00.079 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T20:20:00.166 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T20:20:00.198 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T20:20:00.234 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T20:20:00.262 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T20:20:00.336 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T20:20:00.385 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T20:20:00.462 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T20:20:00.627 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T20:20:00.680 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T20:20:00.975 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T20:20:01.066 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T20:20:01.094 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T20:20:01.127 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T20:20:01.238 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T20:20:01.319 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T20:20:01.354 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T20:20:01.397 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T20:20:01.435 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T20:20:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T20:20:01.477 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T20:20:01.482 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:20:01.571 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:20:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T20:20:01.639 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T20:20:01.652 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T20:20:01.713 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T20:20:01.769 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T20:20:01.843 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T20:20:01.880 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T20:20:01.916 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T20:20:01.972 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T20:20:02.008 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T20:20:02.051 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T20:20:02.093 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T20:20:02.118 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T20:20:02.143 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:20:02.230 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:20:02.238 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T20:20:02.253 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T20:20:02.282 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T20:20:02.311 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T20:20:02.354 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T20:20:02.452 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T20:20:02.486 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T20:20:02.526 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T20:20:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T20:20:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T20:20:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T20:20:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T20:20:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T20:20:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T20:20:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T20:20:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T20:20:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T20:20:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T20:20:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T20:20:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T20:20:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T20:20:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T20:20:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T20:20:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T20:20:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T20:20:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T20:20:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T20:20:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T20:20:04.322 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T20:20:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T20:20:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T20:20:04.512 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T20:20:04.569 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T20:20:04.594 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:20:04.633 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:20:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T20:20:04.704 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T20:20:04.889 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T20:20:04.926 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T20:20:05.007 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T20:20:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:20:05.220 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:20:05.327 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:20:05.327 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T20:20:05.327 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T20:20:05.327 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:05.354 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:20:05.689 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T20:20:05.812 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T20:20:07.111 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T20:20:07.180 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T20:20:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T20:20:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T20:20:07.569 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T20:20:07.612 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:20:07.720 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:20:07.750 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T20:20:07.787 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T20:20:08.292 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T20:20:08.324 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:20:08.387 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:20:08.425 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T20:20:10.743 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:20:10.789 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:20:11.362 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:20:11.400 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:20:11.481 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:20:11.481 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T20:20:11.481 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T20:20:11.481 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:11.518 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:20:16.840 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:20:16.840 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-04-26T20:20:16.840 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-04-26T20:20:16.840 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-04-26T20:20:16.841 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-04-26T20:20:16.841 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-04-26T20:20:16.841 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-04-26T20:20:16.841 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-04-26T20:20:16.841 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:17.015 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:20:17.029 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:20:17.029 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:17.029 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T20:20:17.030 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:18.485 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:20:18.512 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:20:18.512 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:18.512 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T20:20:18.512 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:20:18.512 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:20:18.512 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:18.736 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:20:18.758 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:20:18.758 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:18.758 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T20:20:18.758 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:20:18.758 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:20:18.758 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:20:19.225 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:20:19.225 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:19.225 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T20:20:19.226 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:20:19.226 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:20:19.226 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:19.333 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T20:20:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T20:20:19.437 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T20:20:19.619 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T20:20:19.670 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T20:20:19.747 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T20:20:19.844 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:20:19.983 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:20:20.029 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T20:20:20.061 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T20:20:20.097 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T20:20:20.516 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:20:20.546 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:20:20.578 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T20:20:20.647 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T20:20:20.694 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:20:21.721 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:20:21.754 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:20:22.713 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:20:22.737 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:20:22.839 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:20:23.000 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T20:20:23.032 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:20:23.054 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:20:23.055 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:23.055 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T20:20:23.055 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:20:23.055 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:20:23.055 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:23.099 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:20:23.110 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:20:23.209 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T20:20:23.264 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T20:20:23.544 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:20:23.544 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-04-26T20:20:23.544 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-04-26T20:20:23.544 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-04-26T20:20:23.544 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-04-26T20:20:23.544 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-04-26T20:20:23.544 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-04-26T20:20:23.545 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-04-26T20:20:23.545 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:23.731 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:20:23.747 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:20:23.747 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:23.747 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T20:20:23.747 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:24.190 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T20:20:24.218 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:20:24.241 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:20:24.241 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:24.241 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T20:20:24.241 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:20:24.241 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:20:24.241 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:24.294 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:20:24.315 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:20:24.315 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:24.316 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T20:20:24.316 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:24.369 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:20:24.392 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:20:24.392 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:24.392 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T20:20:24.392 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:20:24.392 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:20:24.392 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:24.703 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:20:24.726 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:20:24.726 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:24.726 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T20:20:24.726 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:20:24.726 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:20:24.727 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:20:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:20:25.311 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:25.311 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T20:20:25.311 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:20:25.311 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:20:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:25.543 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:20:25.566 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:20:25.566 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:25.566 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T20:20:25.566 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:20:25.567 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:20:25.567 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:20:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:20:26.051 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:26.051 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T20:20:26.051 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:20:26.051 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:20:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T20:20:26.216 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T20:20:26.280 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T20:20:26.476 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T20:20:26.528 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T20:20:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T20:20:26.708 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:20:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:20:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T20:20:26.927 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T20:20:26.954 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T20:20:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:20:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:20:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T20:20:27.533 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T20:20:27.559 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:20:28.663 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:20:28.694 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:20:29.282 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T20:20:29.323 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T20:20:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T20:20:29.396 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T20:20:29.423 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T20:20:29.603 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T20:20:29.645 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T20:20:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T20:20:29.715 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:20:29.745 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:20:29.852 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:20:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T20:20:30.047 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:20:30.071 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:20:30.071 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:30.071 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T20:20:30.071 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:20:30.071 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:20:30.071 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:20:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:20:30.235 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T20:20:30.289 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T20:20:30.327 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:20:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:20:30.383 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:20:31.206 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T20:20:31.266 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T20:20:31.292 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:20:31.296 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T20:20:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:20:31.316 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:31.316 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T20:20:31.316 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:20:31.316 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:20:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:31.336 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T20:20:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:20:31.382 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:20:31.382 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:31.383 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T20:20:31.383 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:31.426 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T20:20:31.435 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:20:31.458 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:20:31.459 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:31.459 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T20:20:31.459 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:20:31.459 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:20:31.459 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:31.467 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T20:20:31.467 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:20:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:20:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:20:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:20:31.693 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:20:31.757 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:20:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:20:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:20:31.809 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:20:31.809 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T20:20:31.809 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:20:31.809 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:20:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:36.682 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T20:20:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T20:20:36.814 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T20:20:36.846 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T20:20:36.873 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T20:20:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T20:20:37.110 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T20:20:37.140 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T20:20:37.778 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:20:37.821 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:20:37.841 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:20:38.072 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T20:20:38.073 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T20:20:38.074 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T20:20:38.087 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T20:20:38.087 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T20:20:38.087 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T20:20:38.087 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T20:20:38.087 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T20:20:38.087 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T20:20:38.087 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T20:20:38.087 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T20:20:38.087 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T20:20:38.087 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T20:20:38.088 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T20:20:38.089 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T20:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T20:20:38.091 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T20:20:38.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:20:38.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T20:20:38.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T20:20:38.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T20:20:38.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T20:20:38.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T20:20:38.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T20:20:38.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T20:20:38.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T20:20:38.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T20:20:38.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T20:20:38.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T20:20:38.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T20:20:38.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T20:20:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T20:20:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T20:20:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T20:20:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T20:20:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T20:20:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T20:20:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T20:20:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T20:20:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T20:20:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:20:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T20:20:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T20:20:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T20:20:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T20:20:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T20:20:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T20:20:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:20:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:20:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:20:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:20:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:20:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:20:40.184 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:20:40.184 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:40.184 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-04-26T20:20:40.184 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.184 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.184 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T20:20:40.184 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T20:20:40.184 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-04-26T20:20:40.184 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T20:20:40.185 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T20:20:40.185 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T20:20:40.185 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T20:20:40.185 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.185 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.185 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.185 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.185 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:40.185 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.185 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.185 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T20:20:40.187 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.188 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T20:20:40.189 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.190 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T20:20:40.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T20:20:40.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T20:20:40.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T20:20:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T20:20:40.196 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T20:20:40.196 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T20:20:40.196 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T20:20:40.196 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:40.196 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-04-26T20:20:40.572 DEBUG:teuthology.parallel:result is None 2024-04-26T20:20:44.876 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:20:44.877 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T20:20:44.877 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T20:20:44.877 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T20:20:44.877 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T20:20:44.877 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T20:20:44.877 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T20:20:44.877 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T20:20:44.877 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:20:44.877 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:20:44.877 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T20:20:44.878 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T20:20:44.879 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T20:20:44.879 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T20:20:44.879 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T20:20:44.879 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T20:20:44.891 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T20:20:44.891 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T20:20:44.891 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T20:20:44.891 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T20:20:44.892 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T20:20:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T20:20:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T20:20:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T20:20:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T20:20:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T20:20:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T20:20:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T20:20:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T20:20:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T20:20:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T20:20:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T20:20:44.894 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T20:20:44.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T20:20:44.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T20:20:44.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T20:20:44.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T20:20:44.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:20:44.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T20:20:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:20:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:47.049 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-04-26T20:20:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T20:20:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T20:20:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T20:20:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T20:20:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T20:20:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T20:20:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T20:20:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T20:20:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:20:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T20:20:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T20:20:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T20:20:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T20:20:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T20:20:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T20:20:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T20:20:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:47.061 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-26T20:20:47.453 DEBUG:teuthology.parallel:result is None 2024-04-26T20:20:47.454 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:20:47.682 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T20:20:47.704 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T20:20:47.705 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T20:20:47.705 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T20:20:47.707 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:20:47.950 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T20:20:47.976 INFO:teuthology.orchestra.run.smithi134.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T20:20:47.976 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T20:20:47.977 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T20:20:47.978 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T20:20:47.979 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:20:47.979 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T20:20:48.006 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:20:48.007 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T20:20:48.046 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T20:20:48.046 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:20:48.046 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T20:20:48.071 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T20:20:48.136 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:20:48.136 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T20:20:48.162 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T20:20:48.228 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T20:20:48.228 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:20:48.228 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T20:20:48.253 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T20:20:48.318 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:20:48.318 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T20:20:48.344 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T20:20:48.409 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T20:20:48.410 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:20:48.410 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T20:20:48.435 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T20:20:48.499 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:20:48.499 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T20:20:48.524 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T20:20:48.588 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T20:20:48.673 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T20:20:48.674 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /var/log/ceph 2024-04-26T20:20:48.676 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /var/log/ceph 2024-04-26T20:20:48.700 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T20:20:48.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T20:20:48.741 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T20:20:48.765 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T20:20:48.766 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T20:20:48.808 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T20:20:48.839 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T20:20:48.839 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T20:20:48.839 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T20:20:48.839 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:20:48.839 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T20:20:48.887 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T20:20:48.997 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T20:20:49.022 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:20:49.022 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T20:20:49.037 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T20:20:49.144 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T20:20:49.189 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T20:20:49.189 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T20:20:49.189 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone}}', 'email': None, 'first_in_suite': False, 'job_id': '7675165', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi037.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFJI9/a6xN71scHvRs+a5LoMYoCg+MPzsxjdSAWc0S4CYiqDJV8Blh4sl0X8rm8epzpjduzbDy8WjrR3AzVGZxs=', 'smithi134.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOiFV00u1Fzyoluyqeb6wqgjcP6WsSXZvL9D3fU5DC4maJJdoBJHMG53mJJygw64fB7I3ioVgzYq8DCD6mjgbgA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T20:20:49.189 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T20:20:49.191 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T20:20:49.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T20:20:49.247 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T20:20:49.302 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:20:49.302 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T20:20:49.324 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T20:20:49.325 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-04-26T20:20:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T20:20:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:20:49.380 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T20:20:49.380 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:20:49.380 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:20:49.380 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 20:20:39.718184673 +0000 2024-04-26T20:20:49.380 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 20:18:44.329290614 +0000 2024-04-26T20:20:49.380 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 20:18:44.329290614 +0000 2024-04-26T20:20:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 20:18:44.329290614 +0000 2024-04-26T20:20:49.381 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T20:20:49.442 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:20:49.442 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:20:49.442 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000265532 s, 1.9 MB/s 2024-04-26T20:20:49.444 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T20:20:49.500 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-04-26T20:20:49.555 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T20:20:49.555 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:20:49.555 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T20:20:49.555 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:20:49.555 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:20:49.555 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 20:20:39.718184673 +0000 2024-04-26T20:20:49.555 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 20:18:44.680285901 +0000 2024-04-26T20:20:49.555 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 20:18:44.680285901 +0000 2024-04-26T20:20:49.556 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 20:18:44.680285901 +0000 2024-04-26T20:20:49.556 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T20:20:49.617 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:20:49.618 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:20:49.618 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000191876 s, 2.7 MB/s 2024-04-26T20:20:49.619 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T20:20:49.675 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-04-26T20:20:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T20:20:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:20:49.730 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-26T20:20:49.730 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:20:49.731 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:20:49.731 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 20:20:39.719184660 +0000 2024-04-26T20:20:49.731 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 20:18:45.039281083 +0000 2024-04-26T20:20:49.731 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 20:18:45.039281083 +0000 2024-04-26T20:20:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 20:18:45.039281083 +0000 2024-04-26T20:20:49.731 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T20:20:49.793 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:20:49.794 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:20:49.794 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000197325 s, 2.6 MB/s 2024-04-26T20:20:49.795 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T20:20:49.851 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-04-26T20:20:49.906 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T20:20:49.906 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:20:49.906 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T20:20:49.906 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:20:49.906 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:20:49.906 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 20:20:39.719184660 +0000 2024-04-26T20:20:49.906 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 20:18:45.386276425 +0000 2024-04-26T20:20:49.906 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 20:18:45.386276425 +0000 2024-04-26T20:20:49.906 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 20:18:45.386276425 +0000 2024-04-26T20:20:49.907 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T20:20:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:20:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:20:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000191282 s, 2.7 MB/s 2024-04-26T20:20:49.970 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T20:20:50.027 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:20:50.027 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:20:50.027 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T20:20:50.046 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T20:20:50.046 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-04-26T20:20:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T20:20:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:20:50.103 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-26T20:20:50.103 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:20:50.103 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:20:50.103 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-26 20:20:32.118413381 +0000 2024-04-26T20:20:50.103 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-26 20:18:41.165033529 +0000 2024-04-26T20:20:50.104 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-26 20:18:41.165033529 +0000 2024-04-26T20:20:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-26 20:18:41.165033529 +0000 2024-04-26T20:20:50.104 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T20:20:50.168 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-26T20:20:50.168 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-26T20:20:50.168 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000266984 s, 1.9 MB/s 2024-04-26T20:20:50.169 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T20:20:50.226 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-04-26T20:20:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T20:20:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:20:50.282 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T20:20:50.282 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:20:50.282 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:20:50.282 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-26 20:20:32.118413381 +0000 2024-04-26T20:20:50.282 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-26 20:18:41.504024977 +0000 2024-04-26T20:20:50.282 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-26 20:18:41.504024977 +0000 2024-04-26T20:20:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-26 20:18:41.504024977 +0000 2024-04-26T20:20:50.283 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T20:20:50.345 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-26T20:20:50.345 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-26T20:20:50.345 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000186525 s, 2.7 MB/s 2024-04-26T20:20:50.347 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T20:20:50.404 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-04-26T20:20:50.459 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T20:20:50.460 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:20:50.460 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T20:20:50.460 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:20:50.460 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:20:50.460 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-26 20:20:32.118413381 +0000 2024-04-26T20:20:50.460 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-26 20:18:41.827016828 +0000 2024-04-26T20:20:50.460 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-26 20:18:41.827016828 +0000 2024-04-26T20:20:50.460 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-26 20:18:41.827016828 +0000 2024-04-26T20:20:50.460 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T20:20:50.522 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-26T20:20:50.522 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-26T20:20:50.522 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000248233 s, 2.1 MB/s 2024-04-26T20:20:50.523 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T20:20:50.578 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-04-26T20:20:50.633 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T20:20:50.633 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:20:50.633 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-26T20:20:50.633 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:20:50.633 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:20:50.633 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-26 20:20:32.119413359 +0000 2024-04-26T20:20:50.633 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-26 20:18:42.161008402 +0000 2024-04-26T20:20:50.633 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-26 20:18:42.161008402 +0000 2024-04-26T20:20:50.633 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-26 20:18:42.161008402 +0000 2024-04-26T20:20:50.634 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T20:20:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-26T20:20:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-26T20:20:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000238876 s, 2.1 MB/s 2024-04-26T20:20:50.696 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T20:20:50.751 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:20:50.752 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T20:20:50.752 INFO:tasks.ceph:Generating config... 2024-04-26T20:20:50.753 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T20:20:50.753 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T20:20:50.753 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T20:20:50.753 INFO:tasks.ceph:[client] pid file = 2024-04-26T20:20:50.753 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T20:20:50.754 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T20:20:50.754 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T20:20:50.754 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-04-26T20:20:50.755 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T20:20:50.755 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T20:20:50.755 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T20:20:50.755 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T20:20:50.755 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T20:20:50.755 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T20:20:50.755 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T20:20:50.755 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-26T20:20:50.755 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-26T20:20:50.755 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-26T20:20:50.755 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T20:20:50.755 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T20:20:50.755 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-26T20:20:50.755 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-26T20:20:50.755 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T20:20:50.756 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T20:20:50.756 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-26T20:20:50.756 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T20:20:50.756 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T20:20:50.756 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T20:20:50.756 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T20:20:50.756 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T20:20:50.756 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T20:20:50.756 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T20:20:50.756 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T20:20:50.805 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T20:20:50.808 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T20:20:50.850 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:20:50.914 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.37'), ('mon.b', '172.21.15.134'), ('mon.c', '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]')] 2024-04-26T20:20:50.914 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.37,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T20:20:50.915 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:20:50.915 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T20:20:50.969 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.37 --add b 172.21.15.134 --addv c '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: generated fsid d49b1c18-e1bf-49f2-acda-0904fa7c6684 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 0 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:fsid d49b1c18-e1bf-49f2-acda-0904fa7c6684 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:last_changed 2024-04-26T20:20:51.056339+0000 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-26T20:20:51.056339+0000 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:min_mon_release 17 (quincy) 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:election_strategy: 1 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.c 2024-04-26T20:20:51.056 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T20:20:51.057 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T20:20:51.073 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d49b1c18-e1bf-49f2-acda-0904fa7c6684... 2024-04-26T20:20:51.074 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T20:20:51.115 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T20:20:51.180 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-26T20:20:51.180 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-04-26T20:20:51.180 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T20:20:51.180 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-04-26T20:20:51.180 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.180 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-04-26T20:20:51.180 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.180 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T20:20:51.181 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-04-26T20:20:51.182 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on skipped message = False 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = d49b1c18-e1bf-49f2-acda-0904fa7c6684 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.183 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-04-26T20:20:51.184 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-04-26T20:20:51.184 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.184 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-04-26T20:20:51.184 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-04-26T20:20:51.184 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-04-26T20:20:51.184 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.185 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: bdev async discard = True 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: bdev enable discard = True 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore allocator = bitmap 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 1/20 2024-04-26T20:20:51.186 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 1/20 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 4/10 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-26T20:20:51.187 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: debug finisher = 20 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-04-26T20:20:51.188 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-04-26T20:20:51.189 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:20:51.189 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-04-26T20:20:51.189 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-04-26T20:20:51.189 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:20:51.189 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.189 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T20:20:51.189 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T20:20:51.189 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T20:20:51.189 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.189 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T20:20:51.189 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: mon down mkfs grace = 300 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: mon op complaint time = 120 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-04-26T20:20:51.190 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout: client mount timeout = 600 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-26T20:20:51.191 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-26T20:20:51.192 INFO:teuthology.orchestra.run.smithi037.stdout:[mds] 2024-04-26T20:20:51.192 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds = 20 2024-04-26T20:20:51.192 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds balancer = 20 2024-04-26T20:20:51.192 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-26T20:20:51.192 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug frag = True 2024-04-26T20:20:51.192 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug scatterstat = True 2024-04-26T20:20:51.192 INFO:teuthology.orchestra.run.smithi037.stdout: mds op complaint time = 180 2024-04-26T20:20:51.192 INFO:teuthology.orchestra.run.smithi037.stdout: mds verify scatter = True 2024-04-26T20:20:51.192 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-26T20:20:51.193 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-26T20:20:51.193 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-26T20:20:51.218 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-04-26T20:20:51.218 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-04-26T20:20:51.218 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T20:20:51.218 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-04-26T20:20:51.219 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.219 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-04-26T20:20:51.219 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.220 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-04-26T20:20:51.220 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.220 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-04-26T20:20:51.220 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-04-26T20:20:51.220 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.220 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-04-26T20:20:51.220 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-04-26T20:20:51.220 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.220 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T20:20:51.220 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-04-26T20:20:51.221 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.221 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T20:20:51.221 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T20:20:51.221 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.221 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-04-26T20:20:51.221 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-04-26T20:20:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-04-26T20:20:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-04-26T20:20:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-04-26T20:20:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-04-26T20:20:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-04-26T20:20:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-04-26T20:20:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T20:20:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T20:20:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-04-26T20:20:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T20:20:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-04-26T20:20:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-04-26T20:20:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-04-26T20:20:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-04-26T20:20:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-04-26T20:20:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.37,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-04-26T20:20:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on skipped message = False 2024-04-26T20:20:51.224 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = d49b1c18-e1bf-49f2-acda-0904fa7c6684 2024-04-26T20:20:51.224 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.224 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-04-26T20:20:51.224 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-04-26T20:20:51.224 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.224 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-04-26T20:20:51.224 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-04-26T20:20:51.224 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-04-26T20:20:51.224 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.225 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-04-26T20:20:51.225 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-04-26T20:20:51.225 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.225 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-04-26T20:20:51.225 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-04-26T20:20:51.225 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-04-26T20:20:51.225 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug trim objects = true 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-04-26T20:20:51.226 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.227 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-04-26T20:20:51.227 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-04-26T20:20:51.227 INFO:teuthology.orchestra.run.smithi134.stdout: bdev async discard = True 2024-04-26T20:20:51.227 INFO:teuthology.orchestra.run.smithi134.stdout: bdev enable discard = True 2024-04-26T20:20:51.227 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore allocator = bitmap 2024-04-26T20:20:51.227 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2024-04-26T20:20:51.227 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2024-04-26T20:20:51.227 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 1/20 2024-04-26T20:20:51.228 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 1/20 2024-04-26T20:20:51.228 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-26T20:20:51.228 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-04-26T20:20:51.228 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 4/10 2024-04-26T20:20:51.228 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T20:20:51.228 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2024-04-26T20:20:51.228 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2024-04-26T20:20:51.228 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2024-04-26T20:20:51.228 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2024-04-26T20:20:51.228 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2024-04-26T20:20:51.228 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.229 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-04-26T20:20:51.229 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-26T20:20:51.229 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-04-26T20:20:51.229 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-04-26T20:20:51.229 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-04-26T20:20:51.229 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:20:51.229 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:20:51.229 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-04-26T20:20:51.229 INFO:teuthology.orchestra.run.smithi134.stdout: debug client = 20 2024-04-26T20:20:51.230 INFO:teuthology.orchestra.run.smithi134.stdout: debug finisher = 20 2024-04-26T20:20:51.230 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.230 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-04-26T20:20:51.230 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-26T20:20:51.230 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-04-26T20:20:51.230 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-04-26T20:20:51.230 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-04-26T20:20:51.230 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-04-26T20:20:51.230 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-04-26T20:20:51.230 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:20:51.230 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-04-26T20:20:51.231 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-04-26T20:20:51.231 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:20:51.231 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.231 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T20:20:51.231 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T20:20:51.231 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T20:20:51.231 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.231 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T20:20:51.231 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T20:20:51.231 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: mon down mkfs grace = 300 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: mon op complaint time = 120 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-04-26T20:20:51.232 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout: client mount timeout = 600 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout: debug client = 20 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = "" 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-04-26T20:20:51.233 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-04-26T20:20:51.234 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-04-26T20:20:51.234 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-04-26T20:20:51.234 INFO:teuthology.orchestra.run.smithi134.stdout:[mds] 2024-04-26T20:20:51.234 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds = 20 2024-04-26T20:20:51.234 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds balancer = 20 2024-04-26T20:20:51.234 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-26T20:20:51.234 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug frag = True 2024-04-26T20:20:51.234 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug scatterstat = True 2024-04-26T20:20:51.234 INFO:teuthology.orchestra.run.smithi134.stdout: mds op complaint time = 180 2024-04-26T20:20:51.234 INFO:teuthology.orchestra.run.smithi134.stdout: mds verify scatter = True 2024-04-26T20:20:51.235 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2024-04-26T20:20:51.235 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2024-04-26T20:20:51.235 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2024-04-26T20:20:51.236 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T20:20:51.236 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T20:20:51.290 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T20:20:51.290 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:20:51.290 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T20:20:51.305 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:20:51.306 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T20:20:51.360 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:20:51.360 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:20:51.361 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T20:20:51.361 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:20:51.434 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:20:51.434 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:20:51.489 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:20:51.489 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:20:51.489 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T20:20:51.489 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:20:51.522 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:20:51.522 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:20:51.577 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T20:20:51.577 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T20:20:51.578 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T20:20:51.627 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T20:20:51.629 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T20:20:51.709 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T20:20:51.711 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T20:20:51.712 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T20:20:51.762 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T20:20:51.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T20:20:51.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T20:20:51.878 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T20:20:51.882 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T20:20:51.906 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T20:20:51.955 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T20:20:51.958 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T20:20:51.981 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T20:20:52.070 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T20:20:52.072 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T20:20:52.095 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T20:20:52.096 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T20:20:52.136 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T20:20:52.148 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T20:20:52.229 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T20:20:52.242 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T20:20:52.280 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T20:20:52.292 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T20:20:52.371 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T20:20:52.383 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T20:20:52.383 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T20:20:52.383 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T20:20:52.408 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:20:52.408 INFO:tasks.ceph:role: osd.0 2024-04-26T20:20:52.408 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:20:52.409 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T20:20:52.480 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:20:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:20:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:20:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:20:52.480 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:20:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:20:52.480 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:20:52.480 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:20:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:20:52.480 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:20:52.990 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-26T20:20:52.992 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-26T20:20:52.992 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T20:20:53.143 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T20:20:53.176 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T20:20:53.240 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:20:53.240 INFO:tasks.ceph:role: osd.1 2024-04-26T20:20:53.240 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:20:53.240 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T20:20:53.325 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:20:53.326 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:20:53.326 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:20:53.326 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:20:53.326 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:20:53.326 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:20:53.326 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:20:53.326 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:20:53.326 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:20:53.326 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:20:53.773 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-26T20:20:53.776 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-26T20:20:53.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T20:20:53.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T20:20:53.850 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T20:20:53.914 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:20:53.914 INFO:tasks.ceph:role: osd.2 2024-04-26T20:20:53.914 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:20:53.914 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T20:20:53.986 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:20:53.986 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:20:53.986 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:20:53.987 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:20:53.987 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:20:53.987 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:20:53.987 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:20:53.987 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:20:53.987 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:20:53.987 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:20:54.488 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-26T20:20:54.490 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-26T20:20:54.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T20:20:54.594 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T20:20:54.625 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T20:20:54.689 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:20:54.689 INFO:tasks.ceph:role: osd.3 2024-04-26T20:20:54.689 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:20:54.689 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T20:20:54.762 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:20:54.762 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:20:54.762 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:20:54.762 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:20:54.762 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:20:54.763 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:20:54.763 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:20:54.763 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:20:54.763 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:20:54.763 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:20:55.261 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-26T20:20:55.263 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-26T20:20:55.264 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T20:20:55.390 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T20:20:55.423 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:20:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:55.517+0000 7f308489d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T20:20:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:55.520+0000 7f308489d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T20:20:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:55.520+0000 7f308489d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:20:55.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:55.520+0000 7f308489d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:20:55.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:55.520+0000 7f308489d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:20:55.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:55.528+0000 7f308489d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T20:20:58.378 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T20:20:58.408 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:20:58.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:58.497+0000 7ffb6081c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T20:20:58.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:58.497+0000 7ffb6081c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T20:20:58.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:58.497+0000 7ffb6081c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:20:58.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:58.497+0000 7ffb6081c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:20:58.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:58.497+0000 7ffb6081c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:20:58.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:20:58.498+0000 7ffb6081c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T20:21:01.355 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T20:21:01.384 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:21:01.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:01.474+0000 7f1fcd901640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T20:21:01.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:01.474+0000 7f1fcd901640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T20:21:01.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:01.474+0000 7f1fcd901640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:21:01.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:01.474+0000 7f1fcd901640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:21:01.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:01.474+0000 7f1fcd901640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:21:01.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:01.475+0000 7f1fcd901640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T20:21:04.345 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T20:21:04.375 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:21:04.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:04.464+0000 7f7034c15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T20:21:04.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:04.464+0000 7f7034c15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T20:21:04.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:04.464+0000 7f7034c15640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:21:04.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:04.464+0000 7f7034c15640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:21:04.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:04.464+0000 7f7034c15640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:21:04.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:04.465+0000 7f7034c15640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T20:21:07.309 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T20:21:07.339 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T20:21:07.366 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:21:07.367 INFO:tasks.ceph:role: osd.4 2024-04-26T20:21:07.367 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:21:07.367 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T20:21:07.438 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:21:07.438 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:21:07.438 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:21:07.438 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:21:07.438 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:21:07.439 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:21:07.439 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:21:07.439 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:21:07.439 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:21:07.439 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:21:07.947 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-26T20:21:07.950 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-26T20:21:07.950 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T20:21:08.102 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T20:21:08.135 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T20:21:08.202 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:21:08.202 INFO:tasks.ceph:role: osd.5 2024-04-26T20:21:08.203 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:21:08.203 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T20:21:08.276 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:21:08.276 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:21:08.276 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:21:08.276 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:21:08.276 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:21:08.276 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:21:08.276 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:21:08.276 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:21:08.276 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:21:08.276 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:21:08.729 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-26T20:21:08.732 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-26T20:21:08.732 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T20:21:08.824 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T20:21:08.852 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T20:21:08.917 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:21:08.918 INFO:tasks.ceph:role: osd.6 2024-04-26T20:21:08.918 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:21:08.918 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T20:21:08.989 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:21:08.989 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:21:08.989 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:21:08.989 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:21:08.989 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:21:08.989 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:21:08.989 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:21:08.990 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:21:08.990 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:21:08.990 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:21:09.480 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-26T20:21:09.482 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-26T20:21:09.482 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T20:21:09.594 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T20:21:09.624 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T20:21:09.686 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:21:09.687 INFO:tasks.ceph:role: osd.7 2024-04-26T20:21:09.687 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:21:09.687 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T20:21:09.756 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:21:09.757 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:21:09.757 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:21:09.757 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:21:09.757 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:21:09.757 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:21:09.757 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:21:09.757 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:21:09.757 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:21:09.757 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:21:10.258 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-26T20:21:10.260 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-26T20:21:10.260 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T20:21:10.389 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T20:21:10.420 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:21:10.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:10.515+0000 7ff631701640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T20:21:10.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:10.515+0000 7ff631701640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T20:21:10.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:10.515+0000 7ff631701640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:21:10.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:10.515+0000 7ff631701640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:21:10.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:10.515+0000 7ff631701640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:21:10.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:10.516+0000 7ff631701640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T20:21:13.366 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T20:21:13.394 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:21:13.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:13.483+0000 7fbee5744640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T20:21:13.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:13.484+0000 7fbee5744640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T20:21:13.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:13.484+0000 7fbee5744640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:21:13.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:13.484+0000 7fbee5744640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:21:13.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:13.484+0000 7fbee5744640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:21:13.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:13.484+0000 7fbee5744640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T20:21:16.329 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T20:21:16.356 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:21:16.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:16.444+0000 7f72bdc13640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T20:21:16.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:16.444+0000 7f72bdc13640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T20:21:16.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:16.444+0000 7f72bdc13640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:21:16.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:16.444+0000 7f72bdc13640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:21:16.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:16.444+0000 7f72bdc13640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:21:16.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:16.445+0000 7f72bdc13640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T20:21:19.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:21:19.194 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:21:19.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T20:21:19.330 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:21:19.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:19.421+0000 7ffb1221f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T20:21:19.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:19.422+0000 7ffb1221f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T20:21:19.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:19.422+0000 7ffb1221f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:21:19.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:19.422+0000 7ffb1221f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:21:19.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:19.422+0000 7ffb1221f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:21:19.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:19.422+0000 7ffb1221f640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T20:21:22.287 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T20:21:22.314 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T20:21:22.315 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:22.315 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T20:21:22.345 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:22.345 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T20:21:22.414 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:22.414 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T20:21:22.482 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:22.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T20:21:22.550 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:22.551 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T20:21:22.618 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:22.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T20:21:22.685 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:22.685 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T20:21:22.751 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:22.751 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T20:21:22.780 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:22.780 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T20:21:22.846 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:22.846 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T20:21:22.912 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:22.912 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T20:21:22.978 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:22.978 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T20:21:23.041 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:23.042 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T20:21:23.107 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:23.107 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T20:21:23.171 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:23.171 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T20:21:23.188 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:23.188 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T20:21:23.246 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:23.246 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T20:21:23.261 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:23.261 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T20:21:23.316 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T20:21:23.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T20:21:23.319 DEBUG:teuthology.orchestra.run.smithi134:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T20:21:23.360 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.y] 2024-04-26T20:21:23.360 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCjDCxmUhBnJRAAGsrfntQyYUoHJlj2M2OCVA== 2024-04-26T20:21:23.360 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-26T20:21:23.360 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCjDCxmMZ1xLRAA25Lk5iPjTAlo8+3CnWPavg== 2024-04-26T20:21:23.360 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-26T20:21:23.360 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCjDCxmxYZlNBAAqm/foyZ1j157Yi/vkOspOg== 2024-04-26T20:21:23.360 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.0] 2024-04-26T20:21:23.360 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCnDCxmuavuHhAAHPS8O9ExPpvdHT10wJDHrw== 2024-04-26T20:21:23.360 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.1] 2024-04-26T20:21:23.360 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCqDCxmHDa3HRAAVysGtbYHxPLpHx3kdS1EKw== 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.2] 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCtDCxmwplYHBAAxzI1MLVpn8kK1B27n/ZBoA== 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.3] 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCwDCxmlem9GxAAR6h6rw0owBPPUT/Eyh26bQ== 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.x] 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCjDCxm5VtOKhAA9ZJhTDAGuUn+Un63aSPDbQ== 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCjDCxmSCb5OBAA1qn4Xl59ur/wJDapWSLZ9A== 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCkDCxmx942BBAAJyAAYb/ZMcI0ZYLluqttZw== 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.4] 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQC2DCxmqKnHHhAAkvIA0VP38Y2RYBKMdZFuZA== 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.5] 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQC5DCxmGU7hHBAAAXOqhBDJW/ZiUYlY4dv/DA== 2024-04-26T20:21:23.361 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.6] 2024-04-26T20:21:23.362 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQC8DCxmF7uHGhAAKioy1jI7alaj7WSVHJ1nKA== 2024-04-26T20:21:23.362 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.7] 2024-04-26T20:21:23.362 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQC/DCxmKk8wGRAAaghSXnN4nUlxzhkYGBzZig== 2024-04-26T20:21:23.362 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-26T20:21:23.362 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCkDCxmQTcnCBAAFh1Ew2ihHFtREYikzwF+Dw== 2024-04-26T20:21:23.362 INFO:teuthology.orchestra.run.smithi037.stdout:[client.2] 2024-04-26T20:21:23.362 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCkDCxmUdq2DRAAq/Hml2I/z0bkKXjN9jIeDg== 2024-04-26T20:21:23.362 INFO:teuthology.orchestra.run.smithi037.stdout:[client.1] 2024-04-26T20:21:23.362 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCkDCxmXGS+EBAAvWfY/Xt9yfBLUr3jhc44cA== 2024-04-26T20:21:23.362 INFO:teuthology.orchestra.run.smithi037.stdout:[client.3] 2024-04-26T20:21:23.362 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCkDCxmtrYnFhAADmyeYHIoG22nzUEozY0R9Q== 2024-04-26T20:21:23.378 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.y] 2024-04-26T20:21:23.378 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCjDCxmUhBnJRAAGsrfntQyYUoHJlj2M2OCVA== 2024-04-26T20:21:23.378 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-04-26T20:21:23.378 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCjDCxmMZ1xLRAA25Lk5iPjTAlo8+3CnWPavg== 2024-04-26T20:21:23.378 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-04-26T20:21:23.378 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCjDCxmxYZlNBAAqm/foyZ1j157Yi/vkOspOg== 2024-04-26T20:21:23.379 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.0] 2024-04-26T20:21:23.379 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCnDCxmuavuHhAAHPS8O9ExPpvdHT10wJDHrw== 2024-04-26T20:21:23.379 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.1] 2024-04-26T20:21:23.379 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCqDCxmHDa3HRAAVysGtbYHxPLpHx3kdS1EKw== 2024-04-26T20:21:23.379 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.2] 2024-04-26T20:21:23.379 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCtDCxmwplYHBAAxzI1MLVpn8kK1B27n/ZBoA== 2024-04-26T20:21:23.379 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.3] 2024-04-26T20:21:23.379 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCwDCxmlem9GxAAR6h6rw0owBPPUT/Eyh26bQ== 2024-04-26T20:21:23.379 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.x] 2024-04-26T20:21:23.379 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCjDCxm5VtOKhAA9ZJhTDAGuUn+Un63aSPDbQ== 2024-04-26T20:21:23.379 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-04-26T20:21:23.380 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCjDCxmSCb5OBAA1qn4Xl59ur/wJDapWSLZ9A== 2024-04-26T20:21:23.380 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-04-26T20:21:23.380 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCkDCxmx942BBAAJyAAYb/ZMcI0ZYLluqttZw== 2024-04-26T20:21:23.380 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.4] 2024-04-26T20:21:23.380 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQC2DCxmqKnHHhAAkvIA0VP38Y2RYBKMdZFuZA== 2024-04-26T20:21:23.380 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.5] 2024-04-26T20:21:23.380 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQC5DCxmGU7hHBAAAXOqhBDJW/ZiUYlY4dv/DA== 2024-04-26T20:21:23.380 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.6] 2024-04-26T20:21:23.380 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQC8DCxmF7uHGhAAKioy1jI7alaj7WSVHJ1nKA== 2024-04-26T20:21:23.380 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.7] 2024-04-26T20:21:23.380 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQC/DCxmKk8wGRAAaghSXnN4nUlxzhkYGBzZig== 2024-04-26T20:21:23.381 INFO:teuthology.orchestra.run.smithi134.stdout:[client.0] 2024-04-26T20:21:23.381 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCkDCxmQTcnCBAAFh1Ew2ihHFtREYikzwF+Dw== 2024-04-26T20:21:23.381 INFO:teuthology.orchestra.run.smithi134.stdout:[client.2] 2024-04-26T20:21:23.381 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCkDCxmUdq2DRAAq/Hml2I/z0bkKXjN9jIeDg== 2024-04-26T20:21:23.381 INFO:teuthology.orchestra.run.smithi134.stdout:[client.1] 2024-04-26T20:21:23.381 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCkDCxmXGS+EBAAvWfY/Xt9yfBLUr3jhc44cA== 2024-04-26T20:21:23.381 INFO:teuthology.orchestra.run.smithi134.stdout:[client.3] 2024-04-26T20:21:23.381 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCkDCxmtrYnFhAADmyeYHIoG22nzUEozY0R9Q== 2024-04-26T20:21:23.382 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:21:23.407 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:21:23.471 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:21:23.474 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:21:23.526 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:21:23.528 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:21:23.580 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:23.583 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:23.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:23.634 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:23.683 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:23.685 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:23.733 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:23.735 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:23.784 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:21:23.786 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:21:23.835 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:21:23.838 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:21:23.890 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:21:23.892 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:21:23.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:23.946 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:23.995 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:23.997 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:24.046 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:24.049 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:24.099 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:24.102 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:21:24.150 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:21:24.153 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:21:24.206 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:21:24.209 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:21:24.260 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:21:24.263 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:21:24.314 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:21:24.317 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:21:24.368 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T20:21:24.369 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T20:21:24.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:21:24.489 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T20:21:24.513 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T20:21:24.537 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:21:24.630 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T20:21:24.654 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T20:21:24.717 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:21:24.810 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T20:21:24.833 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:21:24.836 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:21:24.888 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T20:21:24.888 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T20:21:24.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T20:21:24.891 INFO:tasks.ceph.mon.a:Started 2024-04-26T20:21:24.891 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T20:21:24.891 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T20:21:24.931 INFO:tasks.ceph.mon.b:Started 2024-04-26T20:21:24.931 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T20:21:24.931 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T20:21:24.933 INFO:tasks.ceph.mon.c:Started 2024-04-26T20:21:24.933 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T20:21:24.933 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T20:21:24.933 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T20:21:24.935 INFO:tasks.ceph.mgr.y:Started 2024-04-26T20:21:24.935 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T20:21:24.936 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T20:21:24.937 INFO:tasks.ceph.mgr.x:Started 2024-04-26T20:21:24.938 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T20:21:24.938 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T20:21:25.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.038+0000 7f1b7a7e2640 1 Processor -- start 2024-04-26T20:21:25.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.038+0000 7f1b7a7e2640 1 -- start start 2024-04-26T20:21:25.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.038+0000 7f1b7a7e2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1b7415f560 0x7f1b740d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.039+0000 7f1b7a7e2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7415f940 con 0x7f1b740d10a0 2024-04-26T20:21:25.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.039+0000 7f1b7a7e2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b740d43a0 con 0x7f1b740d23a0 2024-04-26T20:21:25.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.039+0000 7f1b7a7e2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1b740d3300 con 0x7f1b7415f560 2024-04-26T20:21:25.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.039+0000 7f1b737fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1b740d10a0 0x7f1b740d1c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49240/0 (socket says 172.21.15.37:49240) 2024-04-26T20:21:25.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.039+0000 7f1b737fe640 1 -- 172.21.15.37:0/3390257535 learned_addr learned my addr 172.21.15.37:0/3390257535 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:25.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.039+0000 7f1b73fff640 1 --2- 172.21.15.37:0/3390257535 >> v2:172.21.15.134:3300/0 conn(0x7f1b7415f560 0x7f1b740d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.039+0000 7f1b73fff640 1 -- 172.21.15.37:0/3390257535 >> v2:172.21.15.134:3300/0 conn(0x7f1b7415f560 msgr2=0x7f1b740d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:21:25.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.039+0000 7f1b73fff640 1 -- 172.21.15.37:0/3390257535 >> v2:172.21.15.134:3300/0 conn(0x7f1b7415f560 msgr2=0x7f1b740d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:21:25.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.039+0000 7f1b73fff640 1 --2- 172.21.15.37:0/3390257535 >> v2:172.21.15.134:3300/0 conn(0x7f1b7415f560 0x7f1b740d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:21:25.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.039+0000 7f1b73fff640 1 --2- 172.21.15.37:0/3390257535 >> v2:172.21.15.134:3300/0 conn(0x7f1b7415f560 0x7f1b740d0b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:21:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.194+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1303256599 0 0) 0x7f1b7415f940 con 0x7f1b740d10a0 2024-04-26T20:21:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.194+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b5c003520 con 0x7f1b740d10a0 2024-04-26T20:21:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.194+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658135628 0 0) 0x7f1b740d43a0 con 0x7f1b740d23a0 2024-04-26T20:21:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.194+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b7415f940 con 0x7f1b740d23a0 2024-04-26T20:21:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.194+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3932432483 0 0) 0x7f1b7415f940 con 0x7f1b740d23a0 2024-04-26T20:21:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.194+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 >> v2:172.21.15.134:3300/0 conn(0x7f1b7415f560 msgr2=0x7f1b740d0b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:21:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.194+0000 7f1b72ffd640 1 --2- 172.21.15.37:0/3390257535 >> v2:172.21.15.134:3300/0 conn(0x7f1b7415f560 0x7f1b740d0b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.194+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 >> v1:172.21.15.37:6789/0 conn(0x7f1b740d10a0 legacy=0x7f1b740d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.194+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b740d35e0 con 0x7f1b740d23a0 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f1b6c0019f0 con 0x7f1b740d23a0 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 >> v1:172.21.15.134:6789/0 conn(0x7f1b740d23a0 legacy=0x7f1b741693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b72ffd640 1 --2- 172.21.15.37:0/3390257535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b5c003b40 0x7f1b5c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b72ffd640 1 --2- 172.21.15.37:0/3390257535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b5c004800 0x7f1b5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b72ffd640 1 --2- 172.21.15.37:0/3390257535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b5c0051c0 0x7f1b5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b740d3300 con 0x7f1b5c0051c0 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c00c0f0 con 0x7f1b5c003b40 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c00c2f0 con 0x7f1b5c004800 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b78d7f640 1 --2- 172.21.15.37:0/3390257535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b5c0051c0 0x7f1b5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b73fff640 1 --2- 172.21.15.37:0/3390257535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b5c003b40 0x7f1b5c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b737fe640 1 --2- 172.21.15.37:0/3390257535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b5c004800 0x7f1b5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b78d7f640 1 -- 172.21.15.37:0/3390257535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b5c004800 msgr2=0x7f1b5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b78d7f640 1 --2- 172.21.15.37:0/3390257535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b5c004800 0x7f1b5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b78d7f640 1 -- 172.21.15.37:0/3390257535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b5c003b40 msgr2=0x7f1b5c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b78d7f640 1 --2- 172.21.15.37:0/3390257535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b5c003b40 0x7f1b5c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.195+0000 7f1b78d7f640 1 -- 172.21.15.37:0/3390257535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7415f940 con 0x7f1b5c0051c0 2024-04-26T20:21:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.196+0000 7f1b78d7f640 1 -- 172.21.15.37:0/3390257535 reap_dead start 2024-04-26T20:21:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.196+0000 7f1b78d7f640 1 --2- 172.21.15.37:0/3390257535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b5c0051c0 0x7f1b5c00b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1b740b4a60 tx=0x7f1b5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25167ead3b53ce59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.197+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c033070 con 0x7f1b5c0051c0 2024-04-26T20:21:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.197+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c02f610 con 0x7f1b5c0051c0 2024-04-26T20:21:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.197+0000 7f1b72ffd640 1 -- 172.21.15.37:0/3390257535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00c0e0 con 0x7f1b5c0051c0 2024-04-26T20:21:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.198+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/3390257535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b5c0051c0 msgr2=0x7f1b5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.198+0000 7f1b7a7e2640 1 --2- 172.21.15.37:0/3390257535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b5c0051c0 0x7f1b5c00b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1b740b4a60 tx=0x7f1b5c005830 comp rx=0 tx=0).stop 2024-04-26T20:21:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.198+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/3390257535 shutdown_connections 2024-04-26T20:21:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.198+0000 7f1b7a7e2640 1 --2- 172.21.15.37:0/3390257535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b5c0051c0 0x7f1b5c00b9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.198+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/3390257535 >> 172.21.15.37:0/3390257535 conn(0x7f1b7409e570 msgr2=0x7f1b740d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.198+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/3390257535 shutdown_connections 2024-04-26T20:21:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.198+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/3390257535 wait complete. 2024-04-26T20:21:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.200+0000 7f1b7a7e2640 1 Processor -- start 2024-04-26T20:21:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.200+0000 7f1b7a7e2640 1 -- start start 2024-04-26T20:21:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.201+0000 7f1b7a7e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b5c0051c0 0x7f1b741f8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.201+0000 7f1b7a7e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ce930 0x7f1b740ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.201+0000 7f1b7a7e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b7415c170 0x7f1b74161b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.201+0000 7f1b7a7e2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b740d3e80 con 0x7f1b7415c170 2024-04-26T20:21:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.201+0000 7f1b7a7e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b740d3ba0 con 0x7f1b740ce930 2024-04-26T20:21:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.201+0000 7f1b7a7e2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1b740d2eb0 con 0x7f1b5c0051c0 2024-04-26T20:21:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.201+0000 7f1b78d7f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b7415c170 0x7f1b74161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.201+0000 7f1b73fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b5c0051c0 0x7f1b741f8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.201+0000 7f1b78d7f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b7415c170 0x7f1b74161b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42346/0 (socket says 172.21.15.37:42346) 2024-04-26T20:21:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.201+0000 7f1b737fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ce930 0x7f1b740ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.201+0000 7f1b78d7f640 1 -- 172.21.15.37:0/1602356938 learned_addr learned my addr 172.21.15.37:0/1602356938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b73fff640 1 -- 172.21.15.37:0/1602356938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ce930 msgr2=0x7f1b740ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b73fff640 1 --2- 172.21.15.37:0/1602356938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ce930 0x7f1b740ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b73fff640 1 -- 172.21.15.37:0/1602356938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b7415c170 msgr2=0x7f1b74161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b73fff640 1 --2- 172.21.15.37:0/1602356938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b7415c170 0x7f1b74161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b73fff640 1 -- 172.21.15.37:0/1602356938 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b740cadd0 con 0x7f1b5c0051c0 2024-04-26T20:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b78d7f640 1 --2- 172.21.15.37:0/1602356938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b7415c170 0x7f1b74161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b73fff640 1 --2- 172.21.15.37:0/1602356938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b5c0051c0 0x7f1b741f8f80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1b640108c0 tx=0x7f1b64010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b70ff9640 1 -- 172.21.15.37:0/1602356938 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6401a020 con 0x7f1b5c0051c0 2024-04-26T20:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b70ff9640 1 -- 172.21.15.37:0/1602356938 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6400e9c0 con 0x7f1b5c0051c0 2024-04-26T20:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/1602356938 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b740cb050 con 0x7f1b5c0051c0 2024-04-26T20:21:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b70ff9640 1 -- 172.21.15.37:0/1602356938 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400ed90 con 0x7f1b5c0051c0 2024-04-26T20:21:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/1602356938 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b741f84a0 con 0x7f1b5c0051c0 2024-04-26T20:21:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b70ff9640 1 -- 172.21.15.37:0/1602356938 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1b6401d020 con 0x7f1b5c0051c0 2024-04-26T20:21:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.202+0000 7f1b70ff9640 1 -- 172.21.15.37:0/1602356938 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1b64014730 con 0x7f1b5c0051c0 2024-04-26T20:21:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.203+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/1602356938 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7406a360 con 0x7f1b5c0051c0 2024-04-26T20:21:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.206+0000 7f1b70ff9640 1 -- 172.21.15.37:0/1602356938 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1b64024020 con 0x7f1b5c0051c0 2024-04-26T20:21:25.300 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:25.299+0000 7f9b0712b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:21:25.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.300+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/1602356938 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1b740d1d30 con 0x7f1b5c0051c0 2024-04-26T20:21:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.304+0000 7f1b70ff9640 1 -- 172.21.15.37:0/1602356938 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f1b740cadd0 con 0x7f1b5c0051c0 2024-04-26T20:21:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-04-26T20:21:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.305+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/1602356938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b5c0051c0 msgr2=0x7f1b741f8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.305+0000 7f1b7a7e2640 1 --2- 172.21.15.37:0/1602356938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b5c0051c0 0x7f1b741f8f80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1b640108c0 tx=0x7f1b64010cc0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.305+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/1602356938 shutdown_connections 2024-04-26T20:21:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.305+0000 7f1b7a7e2640 1 --2- 172.21.15.37:0/1602356938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b7415c170 0x7f1b74161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.305+0000 7f1b7a7e2640 1 --2- 172.21.15.37:0/1602356938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ce930 0x7f1b740ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.305+0000 7f1b7a7e2640 1 --2- 172.21.15.37:0/1602356938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b5c0051c0 0x7f1b741f8f80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.305+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/1602356938 >> 172.21.15.37:0/1602356938 conn(0x7f1b7409e570 msgr2=0x7f1b7415e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.305+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/1602356938 shutdown_connections 2024-04-26T20:21:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:25.305+0000 7f1b7a7e2640 1 -- 172.21.15.37:0/1602356938 wait complete. 2024-04-26T20:21:25.316 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T20:21:25.317 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T20:21:25.317 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:25.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T20:21:25.331 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:25.330+0000 7f567b80f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:21:25.342 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:25.342 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T20:21:25.407 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:25.407 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T20:21:25.458 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:25.458+0000 7f9b0712b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:21:25.471 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:21:25.471 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T20:21:25.487 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:25.486+0000 7f567b80f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:21:25.510 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:25.511 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T20:21:25.511 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:25.510+0000 7f9b0712b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:21:25.535 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:25.535 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T20:21:25.544 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:25.544+0000 7f567b80f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:21:25.599 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:25.599 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T20:21:25.663 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:21:25.663 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T20:21:25.727 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 1b0acb25-b663-44cd-9fec-383b0b04dc80 0 2024-04-26T20:21:25.755 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:25.755+0000 7f9b0712b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:21:25.798 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:25.797+0000 7f567b80f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:21:25.803 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:25.803+0000 7f9b0712b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:21:25.845 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:25.844+0000 7f567b80f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:21:25.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.850+0000 7f6fa2459640 1 Processor -- start 2024-04-26T20:21:25.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.850+0000 7f6fa2459640 1 -- start start 2024-04-26T20:21:25.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6fa2459640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6f9c0b5510 0x7f6f9c0d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6fa2459640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f9c0d3d60 con 0x7f6f9c0b6820 2024-04-26T20:21:25.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6fa2459640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f9c0c9cc0 con 0x7f6f9c0b58f0 2024-04-26T20:21:25.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6fa2459640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6f9c0b5d80 con 0x7f6f9c0b5510 2024-04-26T20:21:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6fa09f6640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f6f9c0b58f0 0x7f6f9c0d3650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:33974/0 (socket says 172.21.15.134:33974) 2024-04-26T20:21:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6fa09f6640 1 -- 172.21.15.134:0/3493274666 learned_addr learned my addr 172.21.15.134:0/3493274666 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-04-26T20:21:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6f9bfff640 1 --2- 172.21.15.134:0/3493274666 >> v2:172.21.15.37:3300/0 conn(0x7f6f9c0b5510 0x7f6f9c0d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6f9affd640 1 -- 172.21.15.134:0/3493274666 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2487147514 0 0) 0x7f6f9c0c9cc0 con 0x7f6f9c0b58f0 2024-04-26T20:21:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6f9affd640 1 -- 172.21.15.134:0/3493274666 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f84003520 con 0x7f6f9c0b58f0 2024-04-26T20:21:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6f9affd640 1 -- 172.21.15.134:0/3493274666 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2800139766 0 0) 0x7f6f9c0d3d60 con 0x7f6f9c0b6820 2024-04-26T20:21:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6f9affd640 1 -- 172.21.15.134:0/3493274666 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f9c0c9cc0 con 0x7f6f9c0b6820 2024-04-26T20:21:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6f9affd640 1 -- 172.21.15.134:0/3493274666 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3035331422 0 0) 0x7f6f84003520 con 0x7f6f9c0b58f0 2024-04-26T20:21:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6f9bfff640 1 -- 172.21.15.134:0/3493274666 >> v1:172.21.15.134:6789/0 conn(0x7f6f9c0b58f0 legacy=0x7f6f9c0d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6f9bfff640 1 -- 172.21.15.134:0/3493274666 >> v1:172.21.15.37:6789/0 conn(0x7f6f9c0b6820 legacy=0x7f6f9c0d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.851+0000 7f6f9bfff640 1 -- 172.21.15.134:0/3493274666 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9c0b5fd0 con 0x7f6f9c0b5510 2024-04-26T20:21:25.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.852+0000 7f6f9bfff640 1 --2- 172.21.15.134:0/3493274666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f9c0b5510 0x7f6f9c0d1890 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0b5f00 tx=0x7f6f8c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d4359db0c7b2be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:25.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.852+0000 7f6f9affd640 1 -- 172.21.15.134:0/3493274666 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c037070 con 0x7f6f9c0b5510 2024-04-26T20:21:25.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.852+0000 7f6f9affd640 1 -- 172.21.15.134:0/3493274666 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f8c02abc0 con 0x7f6f9c0b5510 2024-04-26T20:21:25.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.852+0000 7f6f9affd640 1 -- 172.21.15.134:0/3493274666 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f940019f0 con 0x7f6f9c0b5510 2024-04-26T20:21:25.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.853+0000 7f6fa2459640 1 -- 172.21.15.134:0/3493274666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f9c0b5510 msgr2=0x7f6f9c0d1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.853+0000 7f6fa2459640 1 --2- 172.21.15.134:0/3493274666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f9c0b5510 0x7f6f9c0d1890 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0b5f00 tx=0x7f6f8c02a0b0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.853+0000 7f6fa2459640 1 -- 172.21.15.134:0/3493274666 shutdown_connections 2024-04-26T20:21:25.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.853+0000 7f6fa2459640 1 --2- 172.21.15.134:0/3493274666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f9c0b5510 0x7f6f9c0d1890 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.853+0000 7f6fa2459640 1 -- 172.21.15.134:0/3493274666 >> 172.21.15.134:0/3493274666 conn(0x7f6f9c0d5160 msgr2=0x7f6f9c0d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:25.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.853+0000 7f6fa2459640 1 -- 172.21.15.134:0/3493274666 shutdown_connections 2024-04-26T20:21:25.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.853+0000 7f6fa2459640 1 -- 172.21.15.134:0/3493274666 wait complete. 2024-04-26T20:21:25.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.854+0000 7f6fa2459640 1 Processor -- start 2024-04-26T20:21:25.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.854+0000 7f6fa2459640 1 -- start start 2024-04-26T20:21:25.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa2459640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f9c0b5510 0x7f6f9c158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa2459640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f9c0b58f0 0x7f6f9c0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa2459640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9c0b6820 0x7f6f9c0b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa2459640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c0d0880 con 0x7f6f9c0b5510 2024-04-26T20:21:25.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa2459640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c0d5f60 con 0x7f6f9c0b6820 2024-04-26T20:21:25.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa2459640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6f9c0d0b60 con 0x7f6f9c0b58f0 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa09f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9c0b6820 0x7f6f9c0b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f9c0b58f0 0x7f6f9c0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa09f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9c0b6820 0x7f6f9c0b2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51758/0 (socket says 172.21.15.134:51758) 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa09f6640 1 -- 172.21.15.134:0/2640201936 learned_addr learned my addr 172.21.15.134:0/2640201936 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f9c0b58f0 0x7f6f9c0cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:46280/0 (socket says 172.21.15.134:46280) 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6f9bfff640 1 --2- 172.21.15.134:0/2640201936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f9c0b5510 0x7f6f9c158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa09f6640 1 -- 172.21.15.134:0/2640201936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f9c0b58f0 msgr2=0x7f6f9c0cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa09f6640 1 --2- 172.21.15.134:0/2640201936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f9c0b58f0 0x7f6f9c0cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa09f6640 1 -- 172.21.15.134:0/2640201936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f9c0b5510 msgr2=0x7f6f9c158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa09f6640 1 --2- 172.21.15.134:0/2640201936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f9c0b5510 0x7f6f9c158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6fa09f6640 1 -- 172.21.15.134:0/2640201936 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9c0cdd50 con 0x7f6f9c0b6820 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.855+0000 7f6f9b7fe640 1 --2- 172.21.15.134:0/2640201936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f9c0b58f0 0x7f6f9c0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.856+0000 7f6fa09f6640 1 --2- 172.21.15.134:0/2640201936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9c0b6820 0x7f6f9c0b2de0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400cac0 tx=0x7f6f9400cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.856+0000 7f6f98ff9640 1 -- 172.21.15.134:0/2640201936 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94018070 con 0x7f6f9c0b6820 2024-04-26T20:21:25.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.856+0000 7f6f98ff9640 1 -- 172.21.15.134:0/2640201936 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9400a640 con 0x7f6f9c0b6820 2024-04-26T20:21:25.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.856+0000 7f6f98ff9640 1 -- 172.21.15.134:0/2640201936 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400aa10 con 0x7f6f9c0b6820 2024-04-26T20:21:25.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.856+0000 7f6fa2459640 1 -- 172.21.15.134:0/2640201936 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9c0ce030 con 0x7f6f9c0b6820 2024-04-26T20:21:25.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.856+0000 7f6fa2459640 1 -- 172.21.15.134:0/2640201936 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f9c0ce5f0 con 0x7f6f9c0b6820 2024-04-26T20:21:25.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.856+0000 7f6f98ff9640 1 -- 172.21.15.134:0/2640201936 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6f9401b070 con 0x7f6f9c0b6820 2024-04-26T20:21:25.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.856+0000 7f6f98ff9640 1 -- 172.21.15.134:0/2640201936 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f6f9401d780 con 0x7f6f9c0b6820 2024-04-26T20:21:25.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.857+0000 7f6fa2459640 1 -- 172.21.15.134:0/2640201936 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f9c0693e0 con 0x7f6f9c0b6820 2024-04-26T20:21:25.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.859+0000 7f6f98ff9640 1 -- 172.21.15.134:0/2640201936 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6f9400acf0 con 0x7f6f9c0b6820 2024-04-26T20:21:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.928+0000 7f6fa2459640 1 -- 172.21.15.134:0/2640201936 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1b0acb25-b663-44cd-9fec-383b0b04dc80", "id": 0} v 0) -- 0x7f6f9c15f030 con 0x7f6f9c0b6820 2024-04-26T20:21:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.931+0000 7f6f98ff9640 1 -- 172.21.15.134:0/2640201936 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1b0acb25-b663-44cd-9fec-383b0b04dc80", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f6f9401d9d0 con 0x7f6f9c0b6820 2024-04-26T20:21:25.932 INFO:teuthology.orchestra.run.smithi134.stdout:0 2024-04-26T20:21:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.932+0000 7f6fa2459640 1 -- 172.21.15.134:0/2640201936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9c0b6820 msgr2=0x7f6f9c0b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.932+0000 7f6fa2459640 1 --2- 172.21.15.134:0/2640201936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9c0b6820 0x7f6f9c0b2de0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400cac0 tx=0x7f6f9400cf80 comp rx=0 tx=0).stop 2024-04-26T20:21:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.932+0000 7f6fa2459640 1 -- 172.21.15.134:0/2640201936 shutdown_connections 2024-04-26T20:21:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.932+0000 7f6fa2459640 1 --2- 172.21.15.134:0/2640201936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9c0b6820 0x7f6f9c0b2de0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.932+0000 7f6fa2459640 1 --2- 172.21.15.134:0/2640201936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f9c0b58f0 0x7f6f9c0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.932+0000 7f6fa2459640 1 --2- 172.21.15.134:0/2640201936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f9c0b5510 0x7f6f9c158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.932+0000 7f6fa2459640 1 -- 172.21.15.134:0/2640201936 >> 172.21.15.134:0/2640201936 conn(0x7f6f9c0d5160 msgr2=0x7f6f9c06cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.932+0000 7f6fa2459640 1 -- 172.21.15.134:0/2640201936 shutdown_connections 2024-04-26T20:21:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:25.933+0000 7f6fa2459640 1 -- 172.21.15.134:0/2640201936 wait complete. 2024-04-26T20:21:25.944 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4 1 2024-04-26T20:21:26.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.039+0000 7f492f26b640 1 Processor -- start 2024-04-26T20:21:26.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.039+0000 7f492f26b640 1 -- start start 2024-04-26T20:21:26.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.040+0000 7f492f26b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49280d1260 0x7f49280b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.040+0000 7f492f26b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f492815d700 con 0x7f492815d320 2024-04-26T20:21:26.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.040+0000 7f492f26b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49280ce8f0 con 0x7f49280d16c0 2024-04-26T20:21:26.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.040+0000 7f492f26b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f49280d09a0 con 0x7f49280d1260 2024-04-26T20:21:26.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.040+0000 7f492c806640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49280d1260 0x7f49280b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.040+0000 7f492d808640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f49280d16c0 0x7f49280b3730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:33988/0 (socket says 172.21.15.134:33988) 2024-04-26T20:21:26.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.040+0000 7f492d808640 1 -- 172.21.15.134:0/3482010464 learned_addr learned my addr 172.21.15.134:0/3482010464 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-04-26T20:21:26.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.041+0000 7f491ffff640 1 -- 172.21.15.134:0/3482010464 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3744402367 0 0) 0x7f49280ce8f0 con 0x7f49280d16c0 2024-04-26T20:21:26.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.041+0000 7f491ffff640 1 -- 172.21.15.134:0/3482010464 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4914003520 con 0x7f49280d16c0 2024-04-26T20:21:26.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.041+0000 7f492c806640 1 -- 172.21.15.134:0/3482010464 >> v1:172.21.15.134:6789/0 conn(0x7f49280d16c0 legacy=0x7f49280b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.041+0000 7f492c806640 1 -- 172.21.15.134:0/3482010464 >> v1:172.21.15.37:6789/0 conn(0x7f492815d320 legacy=0x7f49280d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.041+0000 7f492c806640 1 -- 172.21.15.134:0/3482010464 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49280cebd0 con 0x7f49280d1260 2024-04-26T20:21:26.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.041+0000 7f492c806640 1 --2- 172.21.15.134:0/3482010464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49280d1260 0x7f49280b31f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f49240097d0 tx=0x7f492402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d08c82fe2facd4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:26.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.042+0000 7f491ffff640 1 -- 172.21.15.134:0/3482010464 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4924037070 con 0x7f49280d1260 2024-04-26T20:21:26.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.042+0000 7f491ffff640 1 -- 172.21.15.134:0/3482010464 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4924032440 con 0x7f49280d1260 2024-04-26T20:21:26.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.042+0000 7f491ffff640 1 -- 172.21.15.134:0/3482010464 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4924032810 con 0x7f49280d1260 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.042+0000 7f492f26b640 1 -- 172.21.15.134:0/3482010464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49280d1260 msgr2=0x7f49280b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.042+0000 7f492f26b640 1 --2- 172.21.15.134:0/3482010464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49280d1260 0x7f49280b31f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f49240097d0 tx=0x7f492402a180 comp rx=0 tx=0).stop 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.042+0000 7f492f26b640 1 -- 172.21.15.134:0/3482010464 shutdown_connections 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.042+0000 7f492f26b640 1 --2- 172.21.15.134:0/3482010464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49280d1260 0x7f49280b31f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.042+0000 7f492f26b640 1 -- 172.21.15.134:0/3482010464 >> 172.21.15.134:0/3482010464 conn(0x7f49280d2f80 msgr2=0x7f49280d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.042+0000 7f492f26b640 1 -- 172.21.15.134:0/3482010464 shutdown_connections 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.042+0000 7f492f26b640 1 -- 172.21.15.134:0/3482010464 wait complete. 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.043+0000 7f492f26b640 1 Processor -- start 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.043+0000 7f492f26b640 1 -- start start 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492f26b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49280d1260 0x7f49281f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492f26b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49280d16c0 0x7f49280c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492f26b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f492815d320 0x7f49280ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492f26b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f49280b2cd0 con 0x7f492815d320 2024-04-26T20:21:26.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492f26b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f49280ceeb0 con 0x7f49280d1260 2024-04-26T20:21:26.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492f26b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f49280ce550 con 0x7f49280d16c0 2024-04-26T20:21:26.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492d007640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49280d1260 0x7f49281f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492c806640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49280d16c0 0x7f49280c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492d007640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49280d1260 0x7f49281f5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51768/0 (socket says 172.21.15.134:51768) 2024-04-26T20:21:26.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492d007640 1 -- 172.21.15.134:0/2106990542 learned_addr learned my addr 172.21.15.134:0/2106990542 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:26.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492d808640 1 --2- 172.21.15.134:0/2106990542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f492815d320 0x7f49280ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492d007640 1 -- 172.21.15.134:0/2106990542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49280d16c0 msgr2=0x7f49280c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492d007640 1 --2- 172.21.15.134:0/2106990542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49280d16c0 0x7f49280c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492d007640 1 -- 172.21.15.134:0/2106990542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f492815d320 msgr2=0x7f49280ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.044+0000 7f492d007640 1 --2- 172.21.15.134:0/2106990542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f492815d320 0x7f49280ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.045+0000 7f492d007640 1 -- 172.21.15.134:0/2106990542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492815edf0 con 0x7f49280d1260 2024-04-26T20:21:26.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.045+0000 7f492d007640 1 --2- 172.21.15.134:0/2106990542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49280d1260 0x7f49281f5220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f491800c400 tx=0x7f491800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:26.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.045+0000 7f491dffb640 1 -- 172.21.15.134:0/2106990542 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49180052d0 con 0x7f49280d1260 2024-04-26T20:21:26.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.045+0000 7f491dffb640 1 -- 172.21.15.134:0/2106990542 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4918005490 con 0x7f49280d1260 2024-04-26T20:21:26.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.045+0000 7f491dffb640 1 -- 172.21.15.134:0/2106990542 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918007260 con 0x7f49280d1260 2024-04-26T20:21:26.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.045+0000 7f492f26b640 1 -- 172.21.15.134:0/2106990542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492815f0b0 con 0x7f49280d1260 2024-04-26T20:21:26.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.045+0000 7f491dffb640 1 -- 172.21.15.134:0/2106990542 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4918007400 con 0x7f49280d1260 2024-04-26T20:21:26.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.045+0000 7f492f26b640 1 -- 172.21.15.134:0/2106990542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492815f670 con 0x7f49280d1260 2024-04-26T20:21:26.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.045+0000 7f491dffb640 1 -- 172.21.15.134:0/2106990542 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f4918003650 con 0x7f49280d1260 2024-04-26T20:21:26.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.046+0000 7f492f26b640 1 -- 172.21.15.134:0/2106990542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49280693e0 con 0x7f49280d1260 2024-04-26T20:21:26.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.048+0000 7f491dffb640 1 -- 172.21.15.134:0/2106990542 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4918020070 con 0x7f49280d1260 2024-04-26T20:21:26.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.120+0000 7f492f26b640 1 -- 172.21.15.134:0/2106990542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4", "id": 1} v 0) -- 0x7f49280cd670 con 0x7f49280d1260 2024-04-26T20:21:26.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.125+0000 7f491dffb640 1 -- 172.21.15.134:0/2106990542 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f4918003af0 con 0x7f49280d1260 2024-04-26T20:21:26.126 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-04-26T20:21:26.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.126+0000 7f492f26b640 1 -- 172.21.15.134:0/2106990542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49280d1260 msgr2=0x7f49281f5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.126+0000 7f492f26b640 1 --2- 172.21.15.134:0/2106990542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49280d1260 0x7f49281f5220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f491800c400 tx=0x7f491800c8c0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.126+0000 7f492f26b640 1 -- 172.21.15.134:0/2106990542 shutdown_connections 2024-04-26T20:21:26.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.126+0000 7f492f26b640 1 --2- 172.21.15.134:0/2106990542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f492815d320 0x7f49280ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.126+0000 7f492f26b640 1 --2- 172.21.15.134:0/2106990542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49280d16c0 0x7f49280c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.126+0000 7f492f26b640 1 --2- 172.21.15.134:0/2106990542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49280d1260 0x7f49281f5220 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.126+0000 7f492f26b640 1 -- 172.21.15.134:0/2106990542 >> 172.21.15.134:0/2106990542 conn(0x7f49280d2f80 msgr2=0x7f492815c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:26.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.126+0000 7f492f26b640 1 -- 172.21.15.134:0/2106990542 shutdown_connections 2024-04-26T20:21:26.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.126+0000 7f492f26b640 1 -- 172.21.15.134:0/2106990542 wait complete. 2024-04-26T20:21:26.137 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new ac86bc03-5f61-4c8f-bf48-f0de41624cba 2 2024-04-26T20:21:26.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.236+0000 7f398a9e2640 1 Processor -- start 2024-04-26T20:21:26.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.236+0000 7f398a9e2640 1 -- start start 2024-04-26T20:21:26.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f398a9e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f398406fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f398a9e2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f39840730d0 0x7f3984073550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f398a9e2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39840d00a0 con 0x7f39840cfc10 2024-04-26T20:21:26.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f398a9e2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f39840d0380 con 0x7f39840730d0 2024-04-26T20:21:26.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f398a9e2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f39840cd2e0 con 0x7f398415c0d0 2024-04-26T20:21:26.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f3988f7f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f39840730d0 0x7f3984073550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f3983fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f398406fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f3988f7f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f39840730d0 0x7f3984073550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51774/0 (socket says 172.21.15.134:51774) 2024-04-26T20:21:26.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f3983fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f398406fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:46310/0 (socket says 172.21.15.134:46310) 2024-04-26T20:21:26.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f3988f7f640 1 -- 172.21.15.134:0/1804418540 learned_addr learned my addr 172.21.15.134:0/1804418540 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:26.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f3988f7f640 1 -- 172.21.15.134:0/1804418540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 msgr2=0x7f398406fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:26.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f3988f7f640 1 --2- 172.21.15.134:0/1804418540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f398406fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f3988f7f640 1 -- 172.21.15.134:0/1804418540 >> v1:172.21.15.37:6789/0 conn(0x7f39840cfc10 legacy=0x7f39840cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f3988f7f640 1 -- 172.21.15.134:0/1804418540 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39840cd5c0 con 0x7f39840730d0 2024-04-26T20:21:26.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.237+0000 7f3983fff640 1 --2- 172.21.15.134:0/1804418540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f398406fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:26.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.238+0000 7f3988f7f640 1 --2- 172.21.15.134:0/1804418540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39840730d0 0x7f3984073550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f397c009e50 tx=0x7f397c02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b83e20788c95d682 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:26.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.238+0000 7f3982ffd640 1 -- 172.21.15.134:0/1804418540 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c036070 con 0x7f39840730d0 2024-04-26T20:21:26.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.238+0000 7f3982ffd640 1 -- 172.21.15.134:0/1804418540 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c031600 con 0x7f39840730d0 2024-04-26T20:21:26.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.238+0000 7f3982ffd640 1 -- 172.21.15.134:0/1804418540 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0319d0 con 0x7f39840730d0 2024-04-26T20:21:26.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.239+0000 7f398a9e2640 1 -- 172.21.15.134:0/1804418540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39840730d0 msgr2=0x7f3984073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.239+0000 7f398a9e2640 1 --2- 172.21.15.134:0/1804418540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39840730d0 0x7f3984073550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f397c009e50 tx=0x7f397c02a710 comp rx=0 tx=0).stop 2024-04-26T20:21:26.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.239+0000 7f398a9e2640 1 -- 172.21.15.134:0/1804418540 shutdown_connections 2024-04-26T20:21:26.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.239+0000 7f398a9e2640 1 --2- 172.21.15.134:0/1804418540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39840730d0 0x7f3984073550 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.239+0000 7f398a9e2640 1 --2- 172.21.15.134:0/1804418540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f398406fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.239+0000 7f398a9e2640 1 -- 172.21.15.134:0/1804418540 >> 172.21.15.134:0/1804418540 conn(0x7f39840d1e30 msgr2=0x7f39840d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:26.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.239+0000 7f398a9e2640 1 -- 172.21.15.134:0/1804418540 shutdown_connections 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.239+0000 7f398a9e2640 1 -- 172.21.15.134:0/1804418540 wait complete. 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.240+0000 7f398a9e2640 1 Processor -- start 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.240+0000 7f398a9e2640 1 -- start start 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f398a9e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39840730d0 0x7f3984145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f398a9e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f39840b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f398a9e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3984154d60 0x7f398415ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f398a9e2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f398415c4b0 con 0x7f39840730d0 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f398a9e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f39840ccf00 con 0x7f3984154d60 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f398a9e2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f39840b7470 con 0x7f398415c0d0 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f3988f7f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3984154d60 0x7f398415ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f39837fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f39840b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f39837fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f39840b5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:46314/0 (socket says 172.21.15.134:46314) 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f39837fe640 1 -- 172.21.15.134:0/3774759823 learned_addr learned my addr 172.21.15.134:0/3774759823 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f3988f7f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3984154d60 0x7f398415ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51776/0 (socket says 172.21.15.134:51776) 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f3983fff640 1 --2- 172.21.15.134:0/3774759823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39840730d0 0x7f3984145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f39837fe640 1 -- 172.21.15.134:0/3774759823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3984154d60 msgr2=0x7f398415ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f39837fe640 1 --2- 172.21.15.134:0/3774759823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3984154d60 0x7f398415ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f39837fe640 1 -- 172.21.15.134:0/3774759823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39840730d0 msgr2=0x7f3984145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f39837fe640 1 --2- 172.21.15.134:0/3774759823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39840730d0 0x7f3984145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f39837fe640 1 -- 172.21.15.134:0/3774759823 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39840b6860 con 0x7f398415c0d0 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f3988f7f640 1 --2- 172.21.15.134:0/3774759823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3984154d60 0x7f398415ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f39837fe640 1 --2- 172.21.15.134:0/3774759823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f39840b5f60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f397400c430 tx=0x7f397400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.241+0000 7f3980ff9640 1 -- 172.21.15.134:0/3774759823 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39740052d0 con 0x7f398415c0d0 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.242+0000 7f3980ff9640 1 -- 172.21.15.134:0/3774759823 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3974005490 con 0x7f398415c0d0 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.242+0000 7f3980ff9640 1 -- 172.21.15.134:0/3774759823 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974007260 con 0x7f398415c0d0 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.242+0000 7f398a9e2640 1 -- 172.21.15.134:0/3774759823 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39840b6b40 con 0x7f398415c0d0 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.242+0000 7f398a9e2640 1 -- 172.21.15.134:0/3774759823 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39840b7100 con 0x7f398415c0d0 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.242+0000 7f3980ff9640 1 -- 172.21.15.134:0/3774759823 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3974007400 con 0x7f398415c0d0 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.242+0000 7f3980ff9640 1 -- 172.21.15.134:0/3774759823 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f3974011590 con 0x7f398415c0d0 2024-04-26T20:21:26.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.242+0000 7f398a9e2640 1 -- 172.21.15.134:0/3774759823 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39840693e0 con 0x7f398415c0d0 2024-04-26T20:21:26.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.245+0000 7f3980ff9640 1 -- 172.21.15.134:0/3774759823 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3974003a50 con 0x7f398415c0d0 2024-04-26T20:21:26.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.318+0000 7f398a9e2640 1 -- 172.21.15.134:0/3774759823 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ac86bc03-5f61-4c8f-bf48-f0de41624cba", "id": 2} v 0) -- 0x7f39840b22e0 con 0x7f398415c0d0 2024-04-26T20:21:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.324+0000 7f3980ff9640 1 -- 172.21.15.134:0/3774759823 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac86bc03-5f61-4c8f-bf48-f0de41624cba", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f39740035f0 con 0x7f398415c0d0 2024-04-26T20:21:26.324 INFO:teuthology.orchestra.run.smithi134.stdout:2 2024-04-26T20:21:26.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.325+0000 7f398a9e2640 1 -- 172.21.15.134:0/3774759823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 msgr2=0x7f39840b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.325+0000 7f398a9e2640 1 --2- 172.21.15.134:0/3774759823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f39840b5f60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f397400c430 tx=0x7f397400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.325+0000 7f398a9e2640 1 -- 172.21.15.134:0/3774759823 shutdown_connections 2024-04-26T20:21:26.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.325+0000 7f398a9e2640 1 --2- 172.21.15.134:0/3774759823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3984154d60 0x7f398415ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.325+0000 7f398a9e2640 1 --2- 172.21.15.134:0/3774759823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398415c0d0 0x7f39840b5f60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.325+0000 7f398a9e2640 1 --2- 172.21.15.134:0/3774759823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39840730d0 0x7f3984145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.325+0000 7f398a9e2640 1 -- 172.21.15.134:0/3774759823 >> 172.21.15.134:0/3774759823 conn(0x7f39840d1e30 msgr2=0x7f398406ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:26.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.325+0000 7f398a9e2640 1 -- 172.21.15.134:0/3774759823 shutdown_connections 2024-04-26T20:21:26.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.325+0000 7f398a9e2640 1 -- 172.21.15.134:0/3774759823 wait complete. 2024-04-26T20:21:26.336 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 04591f65-3634-49b3-918d-82dfe0c3c61c 3 2024-04-26T20:21:26.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.433+0000 7ff92b95f640 1 Processor -- start 2024-04-26T20:21:26.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.433+0000 7ff92b95f640 1 -- start start 2024-04-26T20:21:26.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.434+0000 7ff92b95f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff92415c0d0 0x7ff92406fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.434+0000 7ff92b95f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240cfc10 0x7ff9240cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.434+0000 7ff92b95f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9240d00a0 con 0x7ff9240ce020 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.434+0000 7ff92b95f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff9240d0380 con 0x7ff92415c0d0 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.434+0000 7ff92b95f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9240cd2e0 con 0x7ff9240cfc10 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.434+0000 7ff928efa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240cfc10 0x7ff9240cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.434+0000 7ff928efa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240cfc10 0x7ff9240cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:46324/0 (socket says 172.21.15.134:46324) 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.434+0000 7ff929efc640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff9240ce020 0x7ff9240730d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:34000/0 (socket says 172.21.15.134:34000) 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.434+0000 7ff928efa640 1 -- 172.21.15.134:0/1796718057 learned_addr learned my addr 172.21.15.134:0/1796718057 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.434+0000 7ff9296fb640 1 --2- 172.21.15.134:0/1796718057 >> v2:172.21.15.37:3300/0 conn(0x7ff92415c0d0 0x7ff92406fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.435+0000 7ff928efa640 1 -- 172.21.15.134:0/1796718057 >> v2:172.21.15.37:3300/0 conn(0x7ff92415c0d0 msgr2=0x7ff92406fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.435+0000 7ff928efa640 1 --2- 172.21.15.134:0/1796718057 >> v2:172.21.15.37:3300/0 conn(0x7ff92415c0d0 0x7ff92406fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.435+0000 7ff917fff640 1 -- 172.21.15.134:0/1796718057 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4075339857 0 0) 0x7ff9240d00a0 con 0x7ff9240ce020 2024-04-26T20:21:26.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.435+0000 7ff928efa640 1 -- 172.21.15.134:0/1796718057 >> v1:172.21.15.134:6789/0 conn(0x7ff9240ce020 legacy=0x7ff9240730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.435+0000 7ff928efa640 1 -- 172.21.15.134:0/1796718057 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9240cd5c0 con 0x7ff9240cfc10 2024-04-26T20:21:26.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.435+0000 7ff928efa640 1 --2- 172.21.15.134:0/1796718057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240cfc10 0x7ff9240cdae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff920009870 tx=0x7ff92002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a11a8a0d26534133 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:26.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.436+0000 7ff917fff640 1 -- 172.21.15.134:0/1796718057 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920037070 con 0x7ff9240cfc10 2024-04-26T20:21:26.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.436+0000 7ff917fff640 1 -- 172.21.15.134:0/1796718057 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff920032440 con 0x7ff9240cfc10 2024-04-26T20:21:26.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.436+0000 7ff917fff640 1 -- 172.21.15.134:0/1796718057 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920032810 con 0x7ff9240cfc10 2024-04-26T20:21:26.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.436+0000 7ff92b95f640 1 -- 172.21.15.134:0/1796718057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240cfc10 msgr2=0x7ff9240cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.436+0000 7ff92b95f640 1 --2- 172.21.15.134:0/1796718057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240cfc10 0x7ff9240cdae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff920009870 tx=0x7ff92002a270 comp rx=0 tx=0).stop 2024-04-26T20:21:26.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.436+0000 7ff92b95f640 1 -- 172.21.15.134:0/1796718057 shutdown_connections 2024-04-26T20:21:26.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.436+0000 7ff92b95f640 1 --2- 172.21.15.134:0/1796718057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240cfc10 0x7ff9240cdae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.436+0000 7ff92b95f640 1 --2- 172.21.15.134:0/1796718057 >> v2:172.21.15.37:3300/0 conn(0x7ff92415c0d0 0x7ff92406fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.436+0000 7ff92b95f640 1 -- 172.21.15.134:0/1796718057 >> 172.21.15.134:0/1796718057 conn(0x7ff9240d1e30 msgr2=0x7ff9240d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:26.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.436+0000 7ff92b95f640 1 -- 172.21.15.134:0/1796718057 shutdown_connections 2024-04-26T20:21:26.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.436+0000 7ff92b95f640 1 -- 172.21.15.134:0/1796718057 wait complete. 2024-04-26T20:21:26.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.437+0000 7ff92b95f640 1 Processor -- start 2024-04-26T20:21:26.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.437+0000 7ff92b95f640 1 -- start start 2024-04-26T20:21:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff92b95f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240ce020 0x7ff92414bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff92b95f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff92415c0d0 0x7ff924159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff92b95f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff92415af50 0x7ff92414f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff92b95f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9240cd080 con 0x7ff92415af50 2024-04-26T20:21:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff92b95f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9240ccf00 con 0x7ff92415c0d0 2024-04-26T20:21:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff92b95f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff924070330 con 0x7ff9240ce020 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff9296fb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240ce020 0x7ff92414bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff928efa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff92415c0d0 0x7ff924159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff9296fb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240ce020 0x7ff92414bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:46332/0 (socket says 172.21.15.134:46332) 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff9296fb640 1 -- 172.21.15.134:0/663095747 learned_addr learned my addr 172.21.15.134:0/663095747 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff929efc640 1 --2- 172.21.15.134:0/663095747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff92415af50 0x7ff92414f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff9296fb640 1 -- 172.21.15.134:0/663095747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff92415c0d0 msgr2=0x7ff924159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff9296fb640 1 --2- 172.21.15.134:0/663095747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff92415c0d0 0x7ff924159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff9296fb640 1 -- 172.21.15.134:0/663095747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff92415af50 msgr2=0x7ff92414f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff9296fb640 1 --2- 172.21.15.134:0/663095747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff92415af50 0x7ff92414f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.438+0000 7ff9296fb640 1 -- 172.21.15.134:0/663095747 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff924159eb0 con 0x7ff9240ce020 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.439+0000 7ff9296fb640 1 --2- 172.21.15.134:0/663095747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240ce020 0x7ff92414bc10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e9b0 tx=0x7ff91800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.439+0000 7ff915ffb640 1 -- 172.21.15.134:0/663095747 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918017070 con 0x7ff9240ce020 2024-04-26T20:21:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.439+0000 7ff915ffb640 1 -- 172.21.15.134:0/663095747 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff91800c9a0 con 0x7ff9240ce020 2024-04-26T20:21:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.439+0000 7ff92b95f640 1 -- 172.21.15.134:0/663095747 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92415a190 con 0x7ff9240ce020 2024-04-26T20:21:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.439+0000 7ff915ffb640 1 -- 172.21.15.134:0/663095747 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91800cd90 con 0x7ff9240ce020 2024-04-26T20:21:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.439+0000 7ff92b95f640 1 -- 172.21.15.134:0/663095747 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92415a750 con 0x7ff9240ce020 2024-04-26T20:21:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.439+0000 7ff915ffb640 1 -- 172.21.15.134:0/663095747 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff91801a070 con 0x7ff9240ce020 2024-04-26T20:21:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.439+0000 7ff915ffb640 1 -- 172.21.15.134:0/663095747 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7ff91801ca20 con 0x7ff9240ce020 2024-04-26T20:21:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.440+0000 7ff92b95f640 1 -- 172.21.15.134:0/663095747 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9240693e0 con 0x7ff9240ce020 2024-04-26T20:21:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.442+0000 7ff915ffb640 1 -- 172.21.15.134:0/663095747 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff91801cd80 con 0x7ff9240ce020 2024-04-26T20:21:26.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.514+0000 7ff92b95f640 1 -- 172.21.15.134:0/663095747 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "04591f65-3634-49b3-918d-82dfe0c3c61c", "id": 3} v 0) -- 0x7ff924073f80 con 0x7ff9240ce020 2024-04-26T20:21:26.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.519+0000 7ff915ffb640 1 -- 172.21.15.134:0/663095747 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "04591f65-3634-49b3-918d-82dfe0c3c61c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7ff91801c660 con 0x7ff9240ce020 2024-04-26T20:21:26.520 INFO:teuthology.orchestra.run.smithi134.stdout:3 2024-04-26T20:21:26.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.520+0000 7ff92b95f640 1 -- 172.21.15.134:0/663095747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240ce020 msgr2=0x7ff92414bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.520+0000 7ff92b95f640 1 --2- 172.21.15.134:0/663095747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240ce020 0x7ff92414bc10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e9b0 tx=0x7ff91800ee70 comp rx=0 tx=0).stop 2024-04-26T20:21:26.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.520+0000 7ff92b95f640 1 -- 172.21.15.134:0/663095747 shutdown_connections 2024-04-26T20:21:26.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.520+0000 7ff92b95f640 1 --2- 172.21.15.134:0/663095747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff92415af50 0x7ff92414f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.520+0000 7ff92b95f640 1 --2- 172.21.15.134:0/663095747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff92415c0d0 0x7ff924159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.520+0000 7ff92b95f640 1 --2- 172.21.15.134:0/663095747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9240ce020 0x7ff92414bc10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.520+0000 7ff92b95f640 1 -- 172.21.15.134:0/663095747 >> 172.21.15.134:0/663095747 conn(0x7ff9240d1e30 msgr2=0x7ff9240b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:26.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.520+0000 7ff92b95f640 1 -- 172.21.15.134:0/663095747 shutdown_connections 2024-04-26T20:21:26.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.520+0000 7ff92b95f640 1 -- 172.21.15.134:0/663095747 wait complete. 2024-04-26T20:21:26.531 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 6099e58a-ea2e-43bf-ac9e-93899bf0093f 4 2024-04-26T20:21:26.546 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:26.545+0000 7f9b0712b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:21:26.579 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:26.578+0000 7f567b80f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:21:26.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.628+0000 7f86c23a5640 1 Processor -- start 2024-04-26T20:21:26.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.628+0000 7f86c23a5640 1 -- start start 2024-04-26T20:21:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86c23a5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f86bc15e150 0x7f86bc161c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86c23a5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc15e530 0x7f86bc0d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86c23a5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86bc06eb10 con 0x7f86bc0d0c90 2024-04-26T20:21:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86c23a5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f86bc06edf0 con 0x7f86bc15e150 2024-04-26T20:21:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86c23a5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc1611b0 con 0x7f86bc15e530 2024-04-26T20:21:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86bbfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f86bc15e150 0x7f86bc161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86bb7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc15e530 0x7f86bc0d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86bbfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f86bc15e150 0x7f86bc161c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51804/0 (socket says 172.21.15.134:51804) 2024-04-26T20:21:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86bb7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc15e530 0x7f86bc0d0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:46342/0 (socket says 172.21.15.134:46342) 2024-04-26T20:21:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86bbfff640 1 -- 172.21.15.134:0/3059327232 learned_addr learned my addr 172.21.15.134:0/3059327232 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86bbfff640 1 -- 172.21.15.134:0/3059327232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc15e530 msgr2=0x7f86bc0d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86bbfff640 1 --2- 172.21.15.134:0/3059327232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc15e530 0x7f86bc0d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86bbfff640 1 -- 172.21.15.134:0/3059327232 >> v1:172.21.15.134:6789/0 conn(0x7f86bc0d0c90 legacy=0x7f86bc06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86bbfff640 1 -- 172.21.15.134:0/3059327232 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc161490 con 0x7f86bc15e150 2024-04-26T20:21:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.629+0000 7f86bb7fe640 1 --2- 172.21.15.134:0/3059327232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc15e530 0x7f86bc0d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:26.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.630+0000 7f86bbfff640 1 --2- 172.21.15.134:0/3059327232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86bc15e150 0x7f86bc161c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009870 tx=0x7f86ac02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d0595143ee90209 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:26.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.630+0000 7f86baffd640 1 -- 172.21.15.134:0/3059327232 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac037070 con 0x7f86bc15e150 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.630+0000 7f86baffd640 1 -- 172.21.15.134:0/3059327232 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac032440 con 0x7f86bc15e150 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.630+0000 7f86baffd640 1 -- 172.21.15.134:0/3059327232 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac032810 con 0x7f86bc15e150 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.630+0000 7f86c23a5640 1 -- 172.21.15.134:0/3059327232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86bc15e150 msgr2=0x7f86bc161c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.630+0000 7f86c23a5640 1 --2- 172.21.15.134:0/3059327232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86bc15e150 0x7f86bc161c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009870 tx=0x7f86ac02a270 comp rx=0 tx=0).stop 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.631+0000 7f86c23a5640 1 -- 172.21.15.134:0/3059327232 shutdown_connections 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.631+0000 7f86c23a5640 1 --2- 172.21.15.134:0/3059327232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc15e530 0x7f86bc0d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.631+0000 7f86c23a5640 1 --2- 172.21.15.134:0/3059327232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86bc15e150 0x7f86bc161c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.631+0000 7f86c23a5640 1 -- 172.21.15.134:0/3059327232 >> 172.21.15.134:0/3059327232 conn(0x7f86bc0ce0b0 msgr2=0x7f86bc0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.631+0000 7f86c23a5640 1 -- 172.21.15.134:0/3059327232 shutdown_connections 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.631+0000 7f86c23a5640 1 -- 172.21.15.134:0/3059327232 wait complete. 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.632+0000 7f86c23a5640 1 Processor -- start 2024-04-26T20:21:26.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.632+0000 7f86c23a5640 1 -- start start 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86c23a5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc0d0c90 0x7f86bc0cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86c23a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86bc15e150 0x7f86bc0c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86c23a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86bc15e530 0x7f86bc0c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86c23a5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc0b0da0 con 0x7f86bc15e150 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86c23a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc06e820 con 0x7f86bc15e530 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86c23a5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc0b0f20 con 0x7f86bc0d0c90 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc0d0c90 0x7f86bc0cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86c0942640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86bc15e530 0x7f86bc0c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc0d0c90 0x7f86bc0cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:46346/0 (socket says 172.21.15.134:46346) 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86bbfff640 1 -- 172.21.15.134:0/1124724434 learned_addr learned my addr 172.21.15.134:0/1124724434 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86bb7fe640 1 --2- 172.21.15.134:0/1124724434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86bc15e150 0x7f86bc0c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86bbfff640 1 -- 172.21.15.134:0/1124724434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86bc15e530 msgr2=0x7f86bc0c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86bbfff640 1 --2- 172.21.15.134:0/1124724434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86bc15e530 0x7f86bc0c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86bbfff640 1 -- 172.21.15.134:0/1124724434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86bc15e150 msgr2=0x7f86bc0c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86bbfff640 1 --2- 172.21.15.134:0/1124724434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86bc15e150 0x7f86bc0c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86bbfff640 1 -- 172.21.15.134:0/1124724434 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc0cb4f0 con 0x7f86bc0d0c90 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86bb7fe640 1 --2- 172.21.15.134:0/1124724434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86bc15e150 0x7f86bc0c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86bbfff640 1 --2- 172.21.15.134:0/1124724434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc0d0c90 0x7f86bc0cccd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f86ac000c00 tx=0x7f86ac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86b8ff9640 1 -- 172.21.15.134:0/1124724434 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac037040 con 0x7f86bc0d0c90 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86b8ff9640 1 -- 172.21.15.134:0/1124724434 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac042070 con 0x7f86bc0d0c90 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86c23a5640 1 -- 172.21.15.134:0/1124724434 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc0ca4b0 con 0x7f86bc0d0c90 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86b8ff9640 1 -- 172.21.15.134:0/1124724434 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac030880 con 0x7f86bc0d0c90 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.633+0000 7f86c23a5640 1 -- 172.21.15.134:0/1124724434 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc0cacf0 con 0x7f86bc0d0c90 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.634+0000 7f86b8ff9640 1 -- 172.21.15.134:0/1124724434 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f86ac030a20 con 0x7f86bc0d0c90 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.634+0000 7f86b8ff9640 1 -- 172.21.15.134:0/1124724434 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f86ac047b40 con 0x7f86bc0d0c90 2024-04-26T20:21:26.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.635+0000 7f86c23a5640 1 -- 172.21.15.134:0/1124724434 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc0693e0 con 0x7f86bc0d0c90 2024-04-26T20:21:26.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.637+0000 7f86b8ff9640 1 -- 172.21.15.134:0/1124724434 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f86bc0ca4b0 con 0x7f86bc0d0c90 2024-04-26T20:21:26.640 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:26.639+0000 7f9b0712b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:21:26.670 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:26.669+0000 7f567b80f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:21:26.691 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:26.691+0000 7f9b0712b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:21:26.715 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:26.714+0000 7f567b80f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:21:26.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.720+0000 7f86c23a5640 1 -- 172.21.15.134:0/1124724434 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6099e58a-ea2e-43bf-ac9e-93899bf0093f", "id": 4} v 0) -- 0x7f86bc0cba60 con 0x7f86bc0d0c90 2024-04-26T20:21:26.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.723+0000 7f86b8ff9640 1 -- 172.21.15.134:0/1124724434 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6099e58a-ea2e-43bf-ac9e-93899bf0093f", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f86ac047dd0 con 0x7f86bc0d0c90 2024-04-26T20:21:26.724 INFO:teuthology.orchestra.run.smithi134.stdout:4 2024-04-26T20:21:26.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.724+0000 7f86c23a5640 1 -- 172.21.15.134:0/1124724434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc0d0c90 msgr2=0x7f86bc0cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.724+0000 7f86c23a5640 1 --2- 172.21.15.134:0/1124724434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc0d0c90 0x7f86bc0cccd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f86ac000c00 tx=0x7f86ac0046c0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.724+0000 7f86c23a5640 1 -- 172.21.15.134:0/1124724434 shutdown_connections 2024-04-26T20:21:26.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.724+0000 7f86c23a5640 1 --2- 172.21.15.134:0/1124724434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86bc15e530 0x7f86bc0c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.724+0000 7f86c23a5640 1 --2- 172.21.15.134:0/1124724434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86bc15e150 0x7f86bc0c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.724+0000 7f86c23a5640 1 --2- 172.21.15.134:0/1124724434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86bc0d0c90 0x7f86bc0cccd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.724+0000 7f86c23a5640 1 -- 172.21.15.134:0/1124724434 >> 172.21.15.134:0/1124724434 conn(0x7f86bc0ce0b0 msgr2=0x7f86bc15efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:26.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.724+0000 7f86c23a5640 1 -- 172.21.15.134:0/1124724434 shutdown_connections 2024-04-26T20:21:26.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.724+0000 7f86c23a5640 1 -- 172.21.15.134:0/1124724434 wait complete. 2024-04-26T20:21:26.735 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 43243462-434b-4689-8988-b2a110153fb1 5 2024-04-26T20:21:26.865 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:26.864+0000 7f9b0712b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:21:26.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.872+0000 7f7c1f114640 1 Processor -- start 2024-04-26T20:21:26.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.872+0000 7f7c1f114640 1 -- start start 2024-04-26T20:21:26.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.872+0000 7f7c1f114640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7c1806c240 0x7f7c18161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.872+0000 7f7c1f114640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c180ce040 con 0x7f7c18073310 2024-04-26T20:21:26.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.873+0000 7f7c1f114640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c180ce2e0 con 0x7f7c180709b0 2024-04-26T20:21:26.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.873+0000 7f7c1f114640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7c180ce5c0 con 0x7f7c1806c240 2024-04-26T20:21:26.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.873+0000 7f7c1ceb0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7c1806c240 0x7f7c18161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.873+0000 7f7c1ceb0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7c1806c240 0x7f7c18161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51818/0 (socket says 172.21.15.134:51818) 2024-04-26T20:21:26.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.873+0000 7f7c1ceb0640 1 -- 172.21.15.134:0/2859488493 learned_addr learned my addr 172.21.15.134:0/2859488493 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:26.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.873+0000 7f7c1ceb0640 1 -- 172.21.15.134:0/2859488493 >> v1:172.21.15.134:6789/0 conn(0x7f7c180709b0 legacy=0x7f7c1806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.873+0000 7f7c1ceb0640 1 -- 172.21.15.134:0/2859488493 >> v1:172.21.15.37:6789/0 conn(0x7f7c18073310 legacy=0x7f7c180737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.873+0000 7f7c1ceb0640 1 -- 172.21.15.134:0/2859488493 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c180cc480 con 0x7f7c1806c240 2024-04-26T20:21:26.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.874+0000 7f7c1ceb0640 1 --2- 172.21.15.134:0/2859488493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c1806c240 0x7f7c18161710 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009870 tx=0x7f7c0c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f823711bdd23d6c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:26.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.874+0000 7f7c177fe640 1 -- 172.21.15.134:0/2859488493 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c037070 con 0x7f7c1806c240 2024-04-26T20:21:26.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.874+0000 7f7c177fe640 1 -- 172.21.15.134:0/2859488493 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c032440 con 0x7f7c1806c240 2024-04-26T20:21:26.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.874+0000 7f7c177fe640 1 -- 172.21.15.134:0/2859488493 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c032810 con 0x7f7c1806c240 2024-04-26T20:21:26.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.874+0000 7f7c1f114640 1 -- 172.21.15.134:0/2859488493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c1806c240 msgr2=0x7f7c18161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.874+0000 7f7c1f114640 1 --2- 172.21.15.134:0/2859488493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c1806c240 0x7f7c18161710 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009870 tx=0x7f7c0c02a270 comp rx=0 tx=0).stop 2024-04-26T20:21:26.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.874+0000 7f7c1f114640 1 -- 172.21.15.134:0/2859488493 shutdown_connections 2024-04-26T20:21:26.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.874+0000 7f7c1f114640 1 --2- 172.21.15.134:0/2859488493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c1806c240 0x7f7c18161710 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.874+0000 7f7c1f114640 1 -- 172.21.15.134:0/2859488493 >> 172.21.15.134:0/2859488493 conn(0x7f7c180d1e30 msgr2=0x7f7c180d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:26.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.875+0000 7f7c1f114640 1 -- 172.21.15.134:0/2859488493 shutdown_connections 2024-04-26T20:21:26.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.875+0000 7f7c1f114640 1 -- 172.21.15.134:0/2859488493 wait complete. 2024-04-26T20:21:26.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.876+0000 7f7c1f114640 1 Processor -- start 2024-04-26T20:21:26.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.876+0000 7f7c1f114640 1 -- start start 2024-04-26T20:21:26.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.876+0000 7f7c1f114640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c1806c240 0x7f7c180d0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.876+0000 7f7c1f114640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c180709b0 0x7f7c18144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.876+0000 7f7c1f114640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c18073310 0x7f7c180cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:26.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.876+0000 7f7c1f114640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c18146d80 con 0x7f7c1806c240 2024-04-26T20:21:26.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.876+0000 7f7c1f114640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c180cca40 con 0x7f7c180709b0 2024-04-26T20:21:26.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.876+0000 7f7c1f114640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7c18146f00 con 0x7f7c18073310 2024-04-26T20:21:26.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c17fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c180709b0 0x7f7c18144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c1d6b1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c18073310 0x7f7c180cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c17fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c180709b0 0x7f7c18144fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51822/0 (socket says 172.21.15.134:51822) 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c17fff640 1 -- 172.21.15.134:0/3767967243 learned_addr learned my addr 172.21.15.134:0/3767967243 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c1ceb0640 1 --2- 172.21.15.134:0/3767967243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c1806c240 0x7f7c180d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c17fff640 1 -- 172.21.15.134:0/3767967243 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c18073310 msgr2=0x7f7c180cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c17fff640 1 --2- 172.21.15.134:0/3767967243 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c18073310 0x7f7c180cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c17fff640 1 -- 172.21.15.134:0/3767967243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c1806c240 msgr2=0x7f7c180d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c17fff640 1 --2- 172.21.15.134:0/3767967243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c1806c240 0x7f7c180d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c17fff640 1 -- 172.21.15.134:0/3767967243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c181f6120 con 0x7f7c180709b0 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c1d6b1640 1 --2- 172.21.15.134:0/3767967243 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c18073310 0x7f7c180cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c17fff640 1 --2- 172.21.15.134:0/3767967243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c180709b0 0x7f7c18144fa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400c450 tx=0x7f7c0400c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c157fa640 1 -- 172.21.15.134:0/3767967243 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c040052d0 con 0x7f7c180709b0 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c157fa640 1 -- 172.21.15.134:0/3767967243 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c04005470 con 0x7f7c180709b0 2024-04-26T20:21:26.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c1f114640 1 -- 172.21.15.134:0/3767967243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c180cfb80 con 0x7f7c180709b0 2024-04-26T20:21:26.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c157fa640 1 -- 172.21.15.134:0/3767967243 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04012870 con 0x7f7c180709b0 2024-04-26T20:21:26.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c1f114640 1 -- 172.21.15.134:0/3767967243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c180d0140 con 0x7f7c180709b0 2024-04-26T20:21:26.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.877+0000 7f7c157fa640 1 -- 172.21.15.134:0/3767967243 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7c04012a10 con 0x7f7c180709b0 2024-04-26T20:21:26.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.878+0000 7f7c157fa640 1 -- 172.21.15.134:0/3767967243 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f7c0401ba20 con 0x7f7c180709b0 2024-04-26T20:21:26.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.879+0000 7f7c1f114640 1 -- 172.21.15.134:0/3767967243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c180d0340 con 0x7f7c180709b0 2024-04-26T20:21:26.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.881+0000 7f7c157fa640 1 -- 172.21.15.134:0/3767967243 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7c180d0340 con 0x7f7c180709b0 2024-04-26T20:21:26.888 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:26.887+0000 7f567b80f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:21:26.917 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:26.917+0000 7f9b0712b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:21:26.938 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:26.937+0000 7f567b80f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:21:26.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.952+0000 7f7c1f114640 1 -- 172.21.15.134:0/3767967243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "43243462-434b-4689-8988-b2a110153fb1", "id": 5} v 0) -- 0x7f7c180693e0 con 0x7f7c180709b0 2024-04-26T20:21:26.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.957+0000 7f7c157fa640 1 -- 172.21.15.134:0/3767967243 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "43243462-434b-4689-8988-b2a110153fb1", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f7c0401bc20 con 0x7f7c180709b0 2024-04-26T20:21:26.958 INFO:teuthology.orchestra.run.smithi134.stdout:5 2024-04-26T20:21:26.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.958+0000 7f7c1f114640 1 -- 172.21.15.134:0/3767967243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c180709b0 msgr2=0x7f7c18144fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:26.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.958+0000 7f7c1f114640 1 --2- 172.21.15.134:0/3767967243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c180709b0 0x7f7c18144fa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400c450 tx=0x7f7c0400c910 comp rx=0 tx=0).stop 2024-04-26T20:21:26.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.958+0000 7f7c1f114640 1 -- 172.21.15.134:0/3767967243 shutdown_connections 2024-04-26T20:21:26.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.958+0000 7f7c1f114640 1 --2- 172.21.15.134:0/3767967243 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c18073310 0x7f7c180cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.958+0000 7f7c1f114640 1 --2- 172.21.15.134:0/3767967243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c180709b0 0x7f7c18144fa0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.958+0000 7f7c1f114640 1 --2- 172.21.15.134:0/3767967243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c1806c240 0x7f7c180d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.958+0000 7f7c1f114640 1 -- 172.21.15.134:0/3767967243 >> 172.21.15.134:0/3767967243 conn(0x7f7c180d1e30 msgr2=0x7f7c180b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:26.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.958+0000 7f7c1f114640 1 -- 172.21.15.134:0/3767967243 shutdown_connections 2024-04-26T20:21:26.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:26.958+0000 7f7c1f114640 1 -- 172.21.15.134:0/3767967243 wait complete. 2024-04-26T20:21:26.969 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 81d64db1-17fb-4555-b606-592c70879aa3 6 2024-04-26T20:21:26.971 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:26.971+0000 7f9b0712b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:21:26.993 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:26.992+0000 7f567b80f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:21:27.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.065+0000 7fba1c646640 1 Processor -- start 2024-04-26T20:21:27.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.065+0000 7fba1c646640 1 -- start start 2024-04-26T20:21:27.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.066+0000 7fba1c646640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fba14168270 0x7fba1416a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.066+0000 7fba1c646640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba14070920 con 0x7fba140b0ff0 2024-04-26T20:21:27.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.066+0000 7fba1c646640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba14070c00 con 0x7fba14070170 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.066+0000 7fba1c646640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fba140cfdf0 con 0x7fba14168270 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.066+0000 7fba1a3e2640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fba14070170 0x7fba1406ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:34032/0 (socket says 172.21.15.134:34032) 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.066+0000 7fba1a3e2640 1 -- 172.21.15.134:0/2986933959 learned_addr learned my addr 172.21.15.134:0/2986933959 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.066+0000 7fba1abe3640 1 --2- 172.21.15.134:0/2986933959 >> v2:172.21.15.37:3300/0 conn(0x7fba14168270 0x7fba1416a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.066+0000 7fba193e0640 1 -- 172.21.15.134:0/2986933959 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4267078160 0 0) 0x7fba14070c00 con 0x7fba14070170 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.066+0000 7fba193e0640 1 -- 172.21.15.134:0/2986933959 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba00003520 con 0x7fba14070170 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.067+0000 7fba193e0640 1 -- 172.21.15.134:0/2986933959 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141520611 0 0) 0x7fba14070920 con 0x7fba140b0ff0 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.067+0000 7fba193e0640 1 -- 172.21.15.134:0/2986933959 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba14070c00 con 0x7fba140b0ff0 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.067+0000 7fba193e0640 1 -- 172.21.15.134:0/2986933959 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1132154307 0 0) 0x7fba00003520 con 0x7fba14070170 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.067+0000 7fba1abe3640 1 -- 172.21.15.134:0/2986933959 >> v1:172.21.15.134:6789/0 conn(0x7fba14070170 legacy=0x7fba1406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.067+0000 7fba1abe3640 1 -- 172.21.15.134:0/2986933959 >> v1:172.21.15.37:6789/0 conn(0x7fba140b0ff0 legacy=0x7fba1406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.067+0000 7fba1abe3640 1 -- 172.21.15.134:0/2986933959 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba14070550 con 0x7fba14168270 2024-04-26T20:21:27.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.067+0000 7fba1abe3640 1 --2- 172.21.15.134:0/2986933959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba14168270 0x7fba1416a660 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009d90 tx=0x7fba0c02a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fd01b5b5035836 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:27.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.068+0000 7fba193e0640 1 -- 172.21.15.134:0/2986933959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04001670 con 0x7fba14168270 2024-04-26T20:21:27.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.068+0000 7fba193e0640 1 -- 172.21.15.134:0/2986933959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba0c007540 con 0x7fba14168270 2024-04-26T20:21:27.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.068+0000 7fba193e0640 1 -- 172.21.15.134:0/2986933959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c007910 con 0x7fba14168270 2024-04-26T20:21:27.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.068+0000 7fba1c646640 1 -- 172.21.15.134:0/2986933959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba14168270 msgr2=0x7fba1416a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.068+0000 7fba1c646640 1 --2- 172.21.15.134:0/2986933959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba14168270 0x7fba1416a660 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009d90 tx=0x7fba0c02a580 comp rx=0 tx=0).stop 2024-04-26T20:21:27.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.068+0000 7fba1c646640 1 -- 172.21.15.134:0/2986933959 shutdown_connections 2024-04-26T20:21:27.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.068+0000 7fba1c646640 1 --2- 172.21.15.134:0/2986933959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba14168270 0x7fba1416a660 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.068+0000 7fba1c646640 1 -- 172.21.15.134:0/2986933959 >> 172.21.15.134:0/2986933959 conn(0x7fba140d1e80 msgr2=0x7fba140d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:27.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.068+0000 7fba1c646640 1 -- 172.21.15.134:0/2986933959 shutdown_connections 2024-04-26T20:21:27.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.068+0000 7fba1c646640 1 -- 172.21.15.134:0/2986933959 wait complete. 2024-04-26T20:21:27.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.069+0000 7fba1c646640 1 Processor -- start 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.069+0000 7fba1c646640 1 -- start start 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.070+0000 7fba1c646640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba14070170 0x7fba140b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.070+0000 7fba1c646640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140b0ff0 0x7fba140c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.070+0000 7fba1c646640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba14168270 0x7fba1415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.070+0000 7fba1c646640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba1406c400 con 0x7fba140b0ff0 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.070+0000 7fba1c646640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fba14070ee0 con 0x7fba14070170 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.070+0000 7fba1c646640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fba1406f510 con 0x7fba14168270 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.070+0000 7fba1a3e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba14070170 0x7fba140b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.070+0000 7fba1abe3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba14168270 0x7fba1415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.070+0000 7fba1a3e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba14070170 0x7fba140b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51838/0 (socket says 172.21.15.134:51838) 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.070+0000 7fba1a3e2640 1 -- 172.21.15.134:0/4100465300 learned_addr learned my addr 172.21.15.134:0/4100465300 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.070+0000 7fba19be1640 1 --2- 172.21.15.134:0/4100465300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140b0ff0 0x7fba140c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba1abe3640 1 -- 172.21.15.134:0/4100465300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba14070170 msgr2=0x7fba140b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba1abe3640 1 --2- 172.21.15.134:0/4100465300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba14070170 0x7fba140b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba1abe3640 1 -- 172.21.15.134:0/4100465300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140b0ff0 msgr2=0x7fba140c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba1abe3640 1 --2- 172.21.15.134:0/4100465300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140b0ff0 0x7fba140c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba1abe3640 1 -- 172.21.15.134:0/4100465300 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba140c6b50 con 0x7fba14168270 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba1a3e2640 1 --2- 172.21.15.134:0/4100465300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba14070170 0x7fba140b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba1abe3640 1 --2- 172.21.15.134:0/4100465300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba14168270 0x7fba1415b1f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fba0c031000 tx=0x7fba0c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba0affd640 1 -- 172.21.15.134:0/4100465300 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c03f070 con 0x7fba14168270 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba0affd640 1 -- 172.21.15.134:0/4100465300 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba0c031bc0 con 0x7fba14168270 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba1c646640 1 -- 172.21.15.134:0/4100465300 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba140c6e30 con 0x7fba14168270 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba0affd640 1 -- 172.21.15.134:0/4100465300 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c007e90 con 0x7fba14168270 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba1c646640 1 -- 172.21.15.134:0/4100465300 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba140c73f0 con 0x7fba14168270 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba0affd640 1 -- 172.21.15.134:0/4100465300 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fba0c0437a0 con 0x7fba14168270 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.071+0000 7fba0affd640 1 -- 172.21.15.134:0/4100465300 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fba0c03bcd0 con 0x7fba14168270 2024-04-26T20:21:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.072+0000 7fba1c646640 1 -- 172.21.15.134:0/4100465300 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba140693e0 con 0x7fba14168270 2024-04-26T20:21:27.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.074+0000 7fba0affd640 1 -- 172.21.15.134:0/4100465300 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fba0c043c20 con 0x7fba14168270 2024-04-26T20:21:27.090 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:27.090+0000 7f9b0712b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:21:27.110 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:27.110+0000 7f567b80f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:21:27.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.146+0000 7fba1c646640 1 -- 172.21.15.134:0/4100465300 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "81d64db1-17fb-4555-b606-592c70879aa3", "id": 6} v 0) -- 0x7fba140b7e00 con 0x7fba14168270 2024-04-26T20:21:27.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.151+0000 7fba0affd640 1 -- 172.21.15.134:0/4100465300 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "81d64db1-17fb-4555-b606-592c70879aa3", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fba0c043e00 con 0x7fba14168270 2024-04-26T20:21:27.152 INFO:teuthology.orchestra.run.smithi134.stdout:6 2024-04-26T20:21:27.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.152+0000 7fba1c646640 1 -- 172.21.15.134:0/4100465300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba14168270 msgr2=0x7fba1415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.152+0000 7fba1c646640 1 --2- 172.21.15.134:0/4100465300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba14168270 0x7fba1415b1f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fba0c031000 tx=0x7fba0c004a40 comp rx=0 tx=0).stop 2024-04-26T20:21:27.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.152+0000 7fba1c646640 1 -- 172.21.15.134:0/4100465300 shutdown_connections 2024-04-26T20:21:27.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.152+0000 7fba1c646640 1 --2- 172.21.15.134:0/4100465300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba14168270 0x7fba1415b1f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.152+0000 7fba1c646640 1 --2- 172.21.15.134:0/4100465300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba140b0ff0 0x7fba140c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.152+0000 7fba1c646640 1 --2- 172.21.15.134:0/4100465300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba14070170 0x7fba140b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.152+0000 7fba1c646640 1 -- 172.21.15.134:0/4100465300 >> 172.21.15.134:0/4100465300 conn(0x7fba140d1e80 msgr2=0x7fba140b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:27.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.152+0000 7fba1c646640 1 -- 172.21.15.134:0/4100465300 shutdown_connections 2024-04-26T20:21:27.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.152+0000 7fba1c646640 1 -- 172.21.15.134:0/4100465300 wait complete. 2024-04-26T20:21:27.163 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new dda9aab8-394e-4948-9c5b-f9cd5f9a792c 7 2024-04-26T20:21:27.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.261+0000 7f9778a99640 1 Processor -- start 2024-04-26T20:21:27.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.261+0000 7f9778a99640 1 -- start start 2024-04-26T20:21:27.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9778a99640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f97741453a0 0x7f9774070940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9778a99640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9774070e80 0x7f9774073080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9778a99640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740b5ba0 0x7f97740735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9778a99640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f97740b4e70 con 0x7f9774070e80 2024-04-26T20:21:27.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9778a99640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9774071400 con 0x7f97741453a0 2024-04-26T20:21:27.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9778a99640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f97740b50c0 con 0x7f97740b5ba0 2024-04-26T20:21:27.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9772d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740b5ba0 0x7f97740735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f977259c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f97741453a0 0x7f9774070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f977259c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f97741453a0 0x7f9774070940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51852/0 (socket says 172.21.15.134:51852) 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9772d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740b5ba0 0x7f97740735c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:46374/0 (socket says 172.21.15.134:46374) 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f977259c640 1 -- 172.21.15.134:0/1140357709 learned_addr learned my addr 172.21.15.134:0/1140357709 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9771d9b640 1 --2- 172.21.15.134:0/1140357709 >> v2:172.21.15.37:3300/0 conn(0x7f9774070e80 0x7f9774073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9772d9d640 1 -- 172.21.15.134:0/1140357709 >> v2:172.21.15.134:3300/0 conn(0x7f97741453a0 msgr2=0x7f9774070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9772d9d640 1 --2- 172.21.15.134:0/1140357709 >> v2:172.21.15.134:3300/0 conn(0x7f97741453a0 0x7f9774070940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9772d9d640 1 -- 172.21.15.134:0/1140357709 >> v2:172.21.15.37:3300/0 conn(0x7f9774070e80 msgr2=0x7f9774073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9772d9d640 1 --2- 172.21.15.134:0/1140357709 >> v2:172.21.15.37:3300/0 conn(0x7f9774070e80 0x7f9774073080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f9772d9d640 1 -- 172.21.15.134:0/1140357709 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97740b53a0 con 0x7f97740b5ba0 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.262+0000 7f977259c640 1 --2- 172.21.15.134:0/1140357709 >> v2:172.21.15.134:3300/0 conn(0x7f97741453a0 0x7f9774070940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:27.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.263+0000 7f9772d9d640 1 --2- 172.21.15.134:0/1140357709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740b5ba0 0x7f97740735c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9774071c30 tx=0x7f976c02e9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1d419f4f0c0142 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.263+0000 7f977159a640 1 -- 172.21.15.134:0/1140357709 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c037070 con 0x7f97740b5ba0 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.263+0000 7f977159a640 1 -- 172.21.15.134:0/1140357709 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976c033850 con 0x7f97740b5ba0 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.263+0000 7f977159a640 1 -- 172.21.15.134:0/1140357709 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c033c20 con 0x7f97740b5ba0 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.263+0000 7f9778a99640 1 -- 172.21.15.134:0/1140357709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740b5ba0 msgr2=0x7f97740735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.263+0000 7f9778a99640 1 --2- 172.21.15.134:0/1140357709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740b5ba0 0x7f97740735c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9774071c30 tx=0x7f976c02e9f0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.264+0000 7f9778a99640 1 -- 172.21.15.134:0/1140357709 shutdown_connections 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.264+0000 7f9778a99640 1 --2- 172.21.15.134:0/1140357709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740b5ba0 0x7f97740735c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.264+0000 7f9778a99640 1 --2- 172.21.15.134:0/1140357709 >> v2:172.21.15.37:3300/0 conn(0x7f9774070e80 0x7f9774073080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.264+0000 7f9778a99640 1 --2- 172.21.15.134:0/1140357709 >> v2:172.21.15.134:3300/0 conn(0x7f97741453a0 0x7f9774070940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.264+0000 7f9778a99640 1 -- 172.21.15.134:0/1140357709 >> 172.21.15.134:0/1140357709 conn(0x7f97740b3b70 msgr2=0x7f97740b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.264+0000 7f9778a99640 1 -- 172.21.15.134:0/1140357709 shutdown_connections 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.264+0000 7f9778a99640 1 -- 172.21.15.134:0/1140357709 wait complete. 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.265+0000 7f9778a99640 1 Processor -- start 2024-04-26T20:21:27.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.265+0000 7f9778a99640 1 -- start start 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9778a99640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9774070e80 0x7f977413f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9778a99640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97740b5ba0 0x7f97741589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9778a99640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97741453a0 0x7f97740cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9778a99640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9774145780 con 0x7f97740b5ba0 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9778a99640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9774145cc0 con 0x7f9774070e80 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9778a99640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f977415fc90 con 0x7f97741453a0 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9772d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97741453a0 0x7f97740cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f977259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9774070e80 0x7f977413f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9772d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97741453a0 0x7f97740cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:46386/0 (socket says 172.21.15.134:46386) 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9772d9d640 1 -- 172.21.15.134:0/1176950823 learned_addr learned my addr 172.21.15.134:0/1176950823 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9771d9b640 1 --2- 172.21.15.134:0/1176950823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97740b5ba0 0x7f97741589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9772d9d640 1 -- 172.21.15.134:0/1176950823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9774070e80 msgr2=0x7f977413f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9772d9d640 1 --2- 172.21.15.134:0/1176950823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9774070e80 0x7f977413f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9772d9d640 1 -- 172.21.15.134:0/1176950823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97740b5ba0 msgr2=0x7f97741589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9772d9d640 1 --2- 172.21.15.134:0/1176950823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97740b5ba0 0x7f97741589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9772d9d640 1 -- 172.21.15.134:0/1176950823 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977413f950 con 0x7f97741453a0 2024-04-26T20:21:27.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9772d9d640 1 --2- 172.21.15.134:0/1176950823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97741453a0 0x7f97740cf800 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f976c009f00 tx=0x7f976c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:27.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9762ffd640 1 -- 172.21.15.134:0/1176950823 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c037040 con 0x7f97741453a0 2024-04-26T20:21:27.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9762ffd640 1 -- 172.21.15.134:0/1176950823 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976c033650 con 0x7f97741453a0 2024-04-26T20:21:27.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9778a99640 1 -- 172.21.15.134:0/1176950823 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977413fbd0 con 0x7f97741453a0 2024-04-26T20:21:27.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.266+0000 7f9762ffd640 1 -- 172.21.15.134:0/1176950823 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c009450 con 0x7f97741453a0 2024-04-26T20:21:27.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.267+0000 7f9778a99640 1 -- 172.21.15.134:0/1176950823 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9774140190 con 0x7f97741453a0 2024-04-26T20:21:27.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.267+0000 7f9762ffd640 1 -- 172.21.15.134:0/1176950823 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f976c03a070 con 0x7f97741453a0 2024-04-26T20:21:27.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.267+0000 7f9762ffd640 1 -- 172.21.15.134:0/1176950823 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f976c04fc70 con 0x7f97741453a0 2024-04-26T20:21:27.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.268+0000 7f9778a99640 1 -- 172.21.15.134:0/1176950823 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97740693e0 con 0x7f97741453a0 2024-04-26T20:21:27.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.270+0000 7f9762ffd640 1 -- 172.21.15.134:0/1176950823 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f977413f950 con 0x7f97741453a0 2024-04-26T20:21:27.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.356+0000 7f9778a99640 1 -- 172.21.15.134:0/1176950823 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dda9aab8-394e-4948-9c5b-f9cd5f9a792c", "id": 7} v 0) -- 0x7f97740b42d0 con 0x7f97741453a0 2024-04-26T20:21:27.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.359+0000 7f9762ffd640 1 -- 172.21.15.134:0/1176950823 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dda9aab8-394e-4948-9c5b-f9cd5f9a792c", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f976c033800 con 0x7f97741453a0 2024-04-26T20:21:27.360 INFO:teuthology.orchestra.run.smithi134.stdout:7 2024-04-26T20:21:27.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.360+0000 7f9778a99640 1 -- 172.21.15.134:0/1176950823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97741453a0 msgr2=0x7f97740cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.360+0000 7f9778a99640 1 --2- 172.21.15.134:0/1176950823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97741453a0 0x7f97740cf800 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f976c009f00 tx=0x7f976c004a40 comp rx=0 tx=0).stop 2024-04-26T20:21:27.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.360+0000 7f9778a99640 1 -- 172.21.15.134:0/1176950823 shutdown_connections 2024-04-26T20:21:27.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.360+0000 7f9778a99640 1 --2- 172.21.15.134:0/1176950823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97741453a0 0x7f97740cf800 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.360+0000 7f9778a99640 1 --2- 172.21.15.134:0/1176950823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97740b5ba0 0x7f97741589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.360+0000 7f9778a99640 1 --2- 172.21.15.134:0/1176950823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9774070e80 0x7f977413f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.360+0000 7f9778a99640 1 -- 172.21.15.134:0/1176950823 >> 172.21.15.134:0/1176950823 conn(0x7f97740b3b70 msgr2=0x7f977406fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:27.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.360+0000 7f9778a99640 1 -- 172.21.15.134:0/1176950823 shutdown_connections 2024-04-26T20:21:27.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:21:27.360+0000 7f9778a99640 1 -- 172.21.15.134:0/1176950823 wait complete. 2024-04-26T20:21:27.372 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T20:21:27.372 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T20:21:27.374 INFO:tasks.ceph.osd.0:Started 2024-04-26T20:21:27.374 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T20:21:27.374 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T20:21:27.378 INFO:tasks.ceph.osd.1:Started 2024-04-26T20:21:27.378 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T20:21:27.378 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T20:21:27.380 INFO:tasks.ceph.osd.2:Started 2024-04-26T20:21:27.381 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T20:21:27.381 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T20:21:27.383 INFO:tasks.ceph.osd.3:Started 2024-04-26T20:21:27.383 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T20:21:27.383 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T20:21:27.385 INFO:tasks.ceph.osd.4:Started 2024-04-26T20:21:27.385 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T20:21:27.385 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T20:21:27.387 INFO:tasks.ceph.osd.5:Started 2024-04-26T20:21:27.388 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T20:21:27.388 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T20:21:27.390 INFO:tasks.ceph.osd.6:Started 2024-04-26T20:21:27.390 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T20:21:27.390 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T20:21:27.392 INFO:tasks.ceph.osd.7:Started 2024-04-26T20:21:27.392 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:21:27.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.522+0000 7f323b807640 1 Processor -- start 2024-04-26T20:21:27.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.523+0000 7f323b807640 1 -- start start 2024-04-26T20:21:27.524 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:27.524+0000 7f9b0712b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:21:27.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.523+0000 7f323b807640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3234073800 0x7f32340b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f323b807640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3234070250 con 0x7f3234144c50 2024-04-26T20:21:27.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f323b807640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3234070530 con 0x7f32340712f0 2024-04-26T20:21:27.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f323b807640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3234070810 con 0x7f3234073800 2024-04-26T20:21:27.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f32395a3640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3234144c50 0x7f32340730f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42164/0 (socket says 172.21.15.37:42164) 2024-04-26T20:21:27.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f32395a3640 1 -- 172.21.15.37:0/2086065566 learned_addr learned my addr 172.21.15.37:0/2086065566 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:27.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f3238da2640 1 --2- 172.21.15.37:0/2086065566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3234073800 0x7f32340b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f322bfff640 1 -- 172.21.15.37:0/2086065566 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2366709227 0 0) 0x7f3234070250 con 0x7f3234144c50 2024-04-26T20:21:27.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f322bfff640 1 -- 172.21.15.37:0/2086065566 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3220003520 con 0x7f3234144c50 2024-04-26T20:21:27.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f322bfff640 1 -- 172.21.15.37:0/2086065566 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512640567 0 0) 0x7f3234070530 con 0x7f32340712f0 2024-04-26T20:21:27.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f3238da2640 1 -- 172.21.15.37:0/2086065566 >> v1:172.21.15.134:6789/0 conn(0x7f32340712f0 legacy=0x7f32340cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f3238da2640 1 -- 172.21.15.37:0/2086065566 >> v1:172.21.15.37:6789/0 conn(0x7f3234144c50 legacy=0x7f32340730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.524+0000 7f3238da2640 1 -- 172.21.15.37:0/2086065566 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3234070af0 con 0x7f3234073800 2024-04-26T20:21:27.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.525+0000 7f3238da2640 1 --2- 172.21.15.37:0/2086065566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3234073800 0x7f32340b6770 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f32240097d0 tx=0x7f322402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f4e1329e0616016 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:27.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.525+0000 7f322bfff640 1 -- 172.21.15.37:0/2086065566 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224037070 con 0x7f3234073800 2024-04-26T20:21:27.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.525+0000 7f322bfff640 1 -- 172.21.15.37:0/2086065566 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3224032440 con 0x7f3234073800 2024-04-26T20:21:27.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.525+0000 7f322bfff640 1 -- 172.21.15.37:0/2086065566 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224032810 con 0x7f3234073800 2024-04-26T20:21:27.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.526+0000 7f323b807640 1 -- 172.21.15.37:0/2086065566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3234073800 msgr2=0x7f32340b6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.526+0000 7f323b807640 1 --2- 172.21.15.37:0/2086065566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3234073800 0x7f32340b6770 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f32240097d0 tx=0x7f322402a210 comp rx=0 tx=0).stop 2024-04-26T20:21:27.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.526+0000 7f323b807640 1 -- 172.21.15.37:0/2086065566 shutdown_connections 2024-04-26T20:21:27.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.526+0000 7f323b807640 1 --2- 172.21.15.37:0/2086065566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3234073800 0x7f32340b6770 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.526+0000 7f323b807640 1 -- 172.21.15.37:0/2086065566 >> 172.21.15.37:0/2086065566 conn(0x7f32340b3b00 msgr2=0x7f32340b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:27.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.526+0000 7f323b807640 1 -- 172.21.15.37:0/2086065566 shutdown_connections 2024-04-26T20:21:27.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.526+0000 7f323b807640 1 -- 172.21.15.37:0/2086065566 wait complete. 2024-04-26T20:21:27.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.527+0000 7f323b807640 1 Processor -- start 2024-04-26T20:21:27.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.527+0000 7f323b807640 1 -- start start 2024-04-26T20:21:27.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f323b807640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32340712f0 0x7f32341401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f323b807640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3234073800 0x7f32340d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f323b807640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3234144c50 0x7f32340d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f323b807640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3234145030 con 0x7f3234073800 2024-04-26T20:21:27.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f323b807640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3234145b00 con 0x7f32340712f0 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f323b807640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f32341451b0 con 0x7f3234144c50 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f3238da2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3234073800 0x7f32340d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f3238da2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3234073800 0x7f32340d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56498/0 (socket says 172.21.15.37:56498) 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f3238da2640 1 -- 172.21.15.37:0/3925937475 learned_addr learned my addr 172.21.15.37:0/3925937475 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f32395a3640 1 --2- 172.21.15.37:0/3925937475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32340712f0 0x7f32341401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f3239da4640 1 --2- 172.21.15.37:0/3925937475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3234144c50 0x7f32340d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f3238da2640 1 -- 172.21.15.37:0/3925937475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3234144c50 msgr2=0x7f32340d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f3238da2640 1 --2- 172.21.15.37:0/3925937475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3234144c50 0x7f32340d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f3238da2640 1 -- 172.21.15.37:0/3925937475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32340712f0 msgr2=0x7f32341401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f3238da2640 1 --2- 172.21.15.37:0/3925937475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32340712f0 0x7f32341401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.528+0000 7f3238da2640 1 -- 172.21.15.37:0/3925937475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3234140720 con 0x7f3234073800 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.529+0000 7f3238da2640 1 --2- 172.21.15.37:0/3925937475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3234073800 0x7f32340d1a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3224035040 tx=0x7f32240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.529+0000 7f3229ffb640 1 -- 172.21.15.37:0/3925937475 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224041070 con 0x7f3234073800 2024-04-26T20:21:27.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.529+0000 7f3229ffb640 1 -- 172.21.15.37:0/3925937475 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3224004220 con 0x7f3234073800 2024-04-26T20:21:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.529+0000 7f3229ffb640 1 -- 172.21.15.37:0/3925937475 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224037040 con 0x7f3234073800 2024-04-26T20:21:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.529+0000 7f323b807640 1 -- 172.21.15.37:0/3925937475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32340c9dc0 con 0x7f3234073800 2024-04-26T20:21:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.529+0000 7f323b807640 1 -- 172.21.15.37:0/3925937475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f323414d890 con 0x7f3234073800 2024-04-26T20:21:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.529+0000 7f3229ffb640 1 -- 172.21.15.37:0/3925937475 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f32240039a0 con 0x7f3234073800 2024-04-26T20:21:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.530+0000 7f3229ffb640 1 -- 172.21.15.37:0/3925937475 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f3224003c90 con 0x7f3234073800 2024-04-26T20:21:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.530+0000 7f323b807640 1 -- 172.21.15.37:0/3925937475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32340693e0 con 0x7f3234073800 2024-04-26T20:21:27.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.534+0000 7f3229ffb640 1 -- 172.21.15.37:0/3925937475 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f322403d4e0 con 0x7f3234073800 2024-04-26T20:21:27.550 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:27.549+0000 7f567b80f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:21:27.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.597+0000 7f323b807640 1 -- 172.21.15.37:0/3925937475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32340740e0 con 0x7f3234073800 2024-04-26T20:21:27.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.598+0000 7f3229ffb640 1 -- 172.21.15.37:0/3925937475 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f322403d6c0 con 0x7f3234073800 2024-04-26T20:21:27.599 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:27.357955+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:21:27.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.599+0000 7f323b807640 1 -- 172.21.15.37:0/3925937475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3234073800 msgr2=0x7f32340d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.599+0000 7f323b807640 1 --2- 172.21.15.37:0/3925937475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3234073800 0x7f32340d1a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3224035040 tx=0x7f32240046c0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.599+0000 7f323b807640 1 -- 172.21.15.37:0/3925937475 shutdown_connections 2024-04-26T20:21:27.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.599+0000 7f323b807640 1 --2- 172.21.15.37:0/3925937475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3234144c50 0x7f32340d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.599+0000 7f323b807640 1 --2- 172.21.15.37:0/3925937475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3234073800 0x7f32340d1a80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.599+0000 7f323b807640 1 --2- 172.21.15.37:0/3925937475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32340712f0 0x7f32341401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.599+0000 7f323b807640 1 -- 172.21.15.37:0/3925937475 >> 172.21.15.37:0/3925937475 conn(0x7f32340b3b00 msgr2=0x7f32340b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:27.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.599+0000 7f323b807640 1 -- 172.21.15.37:0/3925937475 shutdown_connections 2024-04-26T20:21:27.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:27.599+0000 7f323b807640 1 -- 172.21.15.37:0/3925937475 wait complete. 2024-04-26T20:21:27.610 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T20:21:27.611 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T20:21:27.783 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:27.782+0000 7f9b0712b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:21:27.822 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:27.822+0000 7f567b80f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:21:27.833 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:27.833+0000 7f9b0712b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:21:27.877 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:27.876+0000 7f567b80f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:21:27.947 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:27.947+0000 7f9b0712b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:21:27.987 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-26T20:21:27.986+0000 7fc50eedd640 -1 Falling back to public interface 2024-04-26T20:21:27.987 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-26T20:21:27.986+0000 7f1fbe00f640 -1 Falling back to public interface 2024-04-26T20:21:27.991 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-26T20:21:27.990+0000 7f529b7fc640 -1 Falling back to public interface 2024-04-26T20:21:27.993 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-26T20:21:27.992+0000 7fe365a10640 -1 Falling back to public interface 2024-04-26T20:21:28.000 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-26T20:21:27.999+0000 7f559d40d640 -1 Falling back to public interface 2024-04-26T20:21:28.001 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-26T20:21:28.000+0000 7f693a663640 -1 Falling back to public interface 2024-04-26T20:21:28.001 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:28.000+0000 7f567b80f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:21:28.002 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-26T20:21:28.001+0000 7f6a2dfcb640 -1 Falling back to public interface 2024-04-26T20:21:28.003 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-26T20:21:28.002+0000 7f6a2cf7a640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6821/0: (98) Address already in use 2024-04-26T20:21:28.014 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-26T20:21:28.014+0000 7ff0a9814640 -1 Falling back to public interface 2024-04-26T20:21:28.231 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:28.230+0000 7f9b0712b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:21:28.279 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:28.278+0000 7f9b0712b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:21:28.297 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:28.296+0000 7f567b80f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:21:28.347 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:28.346+0000 7f567b80f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:21:28.417 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T20:21:28.434 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:28.433+0000 7f9b0712b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:21:28.509 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.508+0000 7f2428846640 1 Processor -- start 2024-04-26T20:21:28.510 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.508+0000 7f2428846640 1 -- start start 2024-04-26T20:21:28.511 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.509+0000 7f2428846640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f24240d3a10 0x7f24240d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:28.511 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.509+0000 7f2428846640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242415f800 0x7f24240d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:28.511 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.509+0000 7f2428846640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24240b81b0 con 0x7f24240d3df0 2024-04-26T20:21:28.511 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.509+0000 7f2428846640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f24240b8490 con 0x7f24240d3a10 2024-04-26T20:21:28.511 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.509+0000 7f2428846640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f24240b8770 con 0x7f242415f800 2024-04-26T20:21:28.511 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.509+0000 7f2421d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f24240d3df0 0x7f242415f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42166/0 (socket says 172.21.15.37:42166) 2024-04-26T20:21:28.511 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.509+0000 7f2421d9b640 1 -- 172.21.15.37:0/1437882206 learned_addr learned my addr 172.21.15.37:0/1437882206 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:28.512 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.509+0000 7f242259c640 1 --2- 172.21.15.37:0/1437882206 >> v2:172.21.15.134:3300/0 conn(0x7f24240d3a10 0x7f24240d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:28.512 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.509+0000 7f2422d9d640 1 --2- 172.21.15.37:0/1437882206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242415f800 0x7f24240d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:28.512 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.510+0000 7f242159a640 1 -- 172.21.15.37:0/1437882206 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 656909646 0 0) 0x7f24240b81b0 con 0x7f24240d3df0 2024-04-26T20:21:28.512 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.510+0000 7f242159a640 1 -- 172.21.15.37:0/1437882206 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f240c003520 con 0x7f24240d3df0 2024-04-26T20:21:28.512 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.510+0000 7f242259c640 1 -- 172.21.15.37:0/1437882206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242415f800 msgr2=0x7f24240d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:28.512 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.510+0000 7f242259c640 1 --2- 172.21.15.37:0/1437882206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242415f800 0x7f24240d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.512 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.510+0000 7f242259c640 1 -- 172.21.15.37:0/1437882206 >> v1:172.21.15.37:6789/0 conn(0x7f24240d3df0 legacy=0x7f242415f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:28.512 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.510+0000 7f242259c640 1 -- 172.21.15.37:0/1437882206 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24240b8a50 con 0x7f24240d3a10 2024-04-26T20:21:28.513 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.510+0000 7f242259c640 1 --2- 172.21.15.37:0/1437882206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24240d3a10 0x7f24240d0890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f24180097a0 tx=0x7f241802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95a9f4ffbb823710 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:28.513 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.511+0000 7f242159a640 1 -- 172.21.15.37:0/1437882206 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418037070 con 0x7f24240d3a10 2024-04-26T20:21:28.514 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.511+0000 7f242159a640 1 -- 172.21.15.37:0/1437882206 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241802ae50 con 0x7f24240d3a10 2024-04-26T20:21:28.514 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.511+0000 7f242159a640 1 -- 172.21.15.37:0/1437882206 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418032660 con 0x7f24240d3a10 2024-04-26T20:21:28.514 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.511+0000 7f2428846640 1 -- 172.21.15.37:0/1437882206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24240d3a10 msgr2=0x7f24240d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:28.514 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.511+0000 7f2428846640 1 --2- 172.21.15.37:0/1437882206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24240d3a10 0x7f24240d0890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f24180097a0 tx=0x7f241802a100 comp rx=0 tx=0).stop 2024-04-26T20:21:28.514 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.511+0000 7f2428846640 1 -- 172.21.15.37:0/1437882206 shutdown_connections 2024-04-26T20:21:28.514 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.511+0000 7f2428846640 1 --2- 172.21.15.37:0/1437882206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242415f800 0x7f24240d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.514 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.511+0000 7f2428846640 1 --2- 172.21.15.37:0/1437882206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24240d3a10 0x7f24240d0890 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.514 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.511+0000 7f2428846640 1 -- 172.21.15.37:0/1437882206 >> 172.21.15.37:0/1437882206 conn(0x7f242409e520 msgr2=0x7f24240d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:28.514 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.511+0000 7f2428846640 1 -- 172.21.15.37:0/1437882206 shutdown_connections 2024-04-26T20:21:28.514 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.511+0000 7f2428846640 1 -- 172.21.15.37:0/1437882206 wait complete. 2024-04-26T20:21:28.515 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:28.512+0000 7f567b80f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:21:28.517 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.513+0000 7f2428846640 1 Processor -- start 2024-04-26T20:21:28.517 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.513+0000 7f2428846640 1 -- start start 2024-04-26T20:21:28.517 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2428846640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24240d3a10 0x7f24240cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:28.517 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2428846640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24240d3df0 0x7f24240cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:28.517 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2428846640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242415f800 0x7f24240cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:28.517 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2428846640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f24240b5fd0 con 0x7f242415f800 2024-04-26T20:21:28.517 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2428846640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24240b5e50 con 0x7f24240d3df0 2024-04-26T20:21:28.517 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2428846640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f24240b6150 con 0x7f24240d3a10 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2422d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242415f800 0x7f24240cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2422d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242415f800 0x7f24240cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56512/0 (socket says 172.21.15.37:56512) 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2422d9d640 1 -- 172.21.15.37:0/4267225937 learned_addr learned my addr 172.21.15.37:0/4267225937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f242259c640 1 --2- 172.21.15.37:0/4267225937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24240d3a10 0x7f24240cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2421d9b640 1 --2- 172.21.15.37:0/4267225937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24240d3df0 0x7f24240cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2422d9d640 1 -- 172.21.15.37:0/4267225937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24240d3a10 msgr2=0x7f24240cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2422d9d640 1 --2- 172.21.15.37:0/4267225937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24240d3a10 0x7f24240cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2422d9d640 1 -- 172.21.15.37:0/4267225937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24240d3df0 msgr2=0x7f24240cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2422d9d640 1 --2- 172.21.15.37:0/4267225937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24240d3df0 0x7f24240cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2422d9d640 1 -- 172.21.15.37:0/4267225937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24241f50f0 con 0x7f242415f800 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2422d9d640 1 --2- 172.21.15.37:0/4267225937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242415f800 0x7f24240cb260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f241c00efc0 tx=0x7f241c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:28.518 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2412ffd640 1 -- 172.21.15.37:0/4267225937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c018020 con 0x7f242415f800 2024-04-26T20:21:28.519 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2412ffd640 1 -- 172.21.15.37:0/4267225937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c014480 con 0x7f242415f800 2024-04-26T20:21:28.519 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2412ffd640 1 -- 172.21.15.37:0/4267225937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c014870 con 0x7f242415f800 2024-04-26T20:21:28.519 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.514+0000 7f2428846640 1 -- 172.21.15.37:0/4267225937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24240cef60 con 0x7f242415f800 2024-04-26T20:21:28.519 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.515+0000 7f2428846640 1 -- 172.21.15.37:0/4267225937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24241f5930 con 0x7f242415f800 2024-04-26T20:21:28.519 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.515+0000 7f2412ffd640 1 -- 172.21.15.37:0/4267225937 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f241c005d10 con 0x7f242415f800 2024-04-26T20:21:28.519 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.515+0000 7f2412ffd640 1 -- 172.21.15.37:0/4267225937 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f241c013ab0 con 0x7f242415f800 2024-04-26T20:21:28.519 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.515+0000 7f2428846640 1 -- 172.21.15.37:0/4267225937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24240767e0 con 0x7f242415f800 2024-04-26T20:21:28.519 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.518+0000 7f2412ffd640 1 -- 172.21.15.37:0/4267225937 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f241c022070 con 0x7f242415f800 2024-04-26T20:21:28.570 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:28.569+0000 7f9b0712b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:21:28.581 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.580+0000 7f2428846640 1 -- 172.21.15.37:0/4267225937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24240cf510 con 0x7f242415f800 2024-04-26T20:21:28.582 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.581+0000 7f2412ffd640 1 -- 172.21.15.37:0/4267225937 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f24240cef60 con 0x7f242415f800 2024-04-26T20:21:28.582 INFO:teuthology.misc.health.smithi037.stdout: 2024-04-26T20:21:28.582 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":10,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:27.357955+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:21:28.583 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.582+0000 7f2428846640 1 -- 172.21.15.37:0/4267225937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242415f800 msgr2=0x7f24240cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:28.583 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.582+0000 7f2428846640 1 --2- 172.21.15.37:0/4267225937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242415f800 0x7f24240cb260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f241c00efc0 tx=0x7f241c00c490 comp rx=0 tx=0).stop 2024-04-26T20:21:28.583 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.582+0000 7f2428846640 1 -- 172.21.15.37:0/4267225937 shutdown_connections 2024-04-26T20:21:28.583 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.582+0000 7f2428846640 1 --2- 172.21.15.37:0/4267225937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242415f800 0x7f24240cb260 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.584 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.582+0000 7f2428846640 1 --2- 172.21.15.37:0/4267225937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24240d3df0 0x7f24240cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.584 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.582+0000 7f2428846640 1 --2- 172.21.15.37:0/4267225937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24240d3a10 0x7f24240cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.584 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.582+0000 7f2428846640 1 -- 172.21.15.37:0/4267225937 >> 172.21.15.37:0/4267225937 conn(0x7f242409e520 msgr2=0x7f242415e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:28.584 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.582+0000 7f2428846640 1 -- 172.21.15.37:0/4267225937 shutdown_connections 2024-04-26T20:21:28.584 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:28.582+0000 7f2428846640 1 -- 172.21.15.37:0/4267225937 wait complete. 2024-04-26T20:21:28.592 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T20:21:28.620 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:28.619+0000 7f9b0712b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:21:28.658 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:28.657+0000 7f567b80f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:21:28.688 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:28.687+0000 7f9b0712b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:21:28.710 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:28.709+0000 7f567b80f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:21:28.732 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:28.732+0000 7f9b0712b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:21:28.783 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:28.782+0000 7f567b80f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:21:28.831 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:28.830+0000 7f567b80f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:21:29.207 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:29.206+0000 7f9b0712b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:21:29.321 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:29.321+0000 7f9b0712b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:21:29.335 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:29.334+0000 7f567b80f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:21:29.458 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-26T20:21:29.457+0000 7f567b80f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:21:30.337 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:30.337+0000 7f9aa72b5640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T20:21:30.338 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:21:30.337+0000 7f9aa72b5640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T20:21:31.488 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-26T20:21:31.487+0000 7f1fbe00f640 -1 osd.1 0 log_to_monitors true 2024-04-26T20:21:31.489 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-26T20:21:31.488+0000 7f559d40d640 -1 osd.6 0 log_to_monitors true 2024-04-26T20:21:31.509 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-26T20:21:31.508+0000 7fc50eedd640 -1 osd.7 0 log_to_monitors true 2024-04-26T20:21:31.510 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-26T20:21:31.509+0000 7fe365a10640 -1 osd.0 0 log_to_monitors true 2024-04-26T20:21:31.515 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-26T20:21:31.514+0000 7f693a663640 -1 osd.2 0 log_to_monitors true 2024-04-26T20:21:31.515 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-26T20:21:31.515+0000 7f529b7fc640 -1 osd.5 0 log_to_monitors true 2024-04-26T20:21:31.536 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-26T20:21:31.535+0000 7f6a2dfcb640 -1 osd.3 0 log_to_monitors true 2024-04-26T20:21:31.538 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-26T20:21:31.538+0000 7ff0a9814640 -1 osd.4 0 log_to_monitors true 2024-04-26T20:21:35.399 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T20:21:35.402 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-26T20:21:35.400+0000 7ff0a5259640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T20:21:35.410 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-26T20:21:35.409+0000 7ff09d079640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:21:35.416 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-26T20:21:35.415+0000 7f5598e38640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T20:21:35.426 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-26T20:21:35.425+0000 7f5590c58640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:21:35.427 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-26T20:21:35.426+0000 7f5297792640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T20:21:35.433 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-26T20:21:35.431+0000 7fc50ae73640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T20:21:35.442 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-26T20:21:35.441+0000 7f528f5b2640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:21:35.444 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-26T20:21:35.443+0000 7f69365f9640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T20:21:35.445 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-26T20:21:35.444+0000 7fc502c93640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:21:35.462 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-26T20:21:35.462+0000 7f6a29f61640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T20:21:35.467 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-26T20:21:35.466+0000 7f692e419640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:21:35.475 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-26T20:21:35.474+0000 7f6a21d81640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:21:35.476 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-26T20:21:35.475+0000 7f1fb9a22640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T20:21:35.487 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-26T20:21:35.487+0000 7f1fb1842640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:21:35.503 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-26T20:21:35.503+0000 7fe361304640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T20:21:35.514 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-26T20:21:35.513+0000 7fe359124640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:21:35.530 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.529+0000 7f6ddd15f640 1 Processor -- start 2024-04-26T20:21:35.530 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.529+0000 7f6ddd15f640 1 -- start start 2024-04-26T20:21:35.538 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.529+0000 7f6ddd15f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6dd815d170 0x7f6dd80cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.529+0000 7f6ddd15f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6dd80ce700 0x7f6dd8070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.529+0000 7f6ddd15f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dd80d0be0 con 0x7f6dd80b5770 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.529+0000 7f6ddd15f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd809ca50 con 0x7f6dd80ce700 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.529+0000 7f6ddd15f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd80d0ec0 con 0x7f6dd815d170 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.530+0000 7f6dd759e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6dd80ce700 0x7f6dd8070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.530+0000 7f6dd659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6dd80b5770 0x7f6dd80cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42174/0 (socket says 172.21.15.37:42174) 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.530+0000 7f6dd659c640 1 -- 172.21.15.37:0/2915570085 learned_addr learned my addr 172.21.15.37:0/2915570085 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.530+0000 7f6dd6d9d640 1 --2- 172.21.15.37:0/2915570085 >> v2:172.21.15.134:3300/0 conn(0x7f6dd815d170 0x7f6dd80cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.530+0000 7f6dd759e640 1 -- 172.21.15.37:0/2915570085 >> v2:172.21.15.134:3300/0 conn(0x7f6dd815d170 msgr2=0x7f6dd80cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.530+0000 7f6dd759e640 1 --2- 172.21.15.37:0/2915570085 >> v2:172.21.15.134:3300/0 conn(0x7f6dd815d170 0x7f6dd80cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.530+0000 7f6dd759e640 1 -- 172.21.15.37:0/2915570085 >> v1:172.21.15.37:6789/0 conn(0x7f6dd80b5770 legacy=0x7f6dd80cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:35.539 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.530+0000 7f6dd759e640 1 -- 172.21.15.37:0/2915570085 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd80d11a0 con 0x7f6dd80ce700 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.530+0000 7f6dd5d9b640 1 -- 172.21.15.37:0/2915570085 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481714419 0 0) 0x7f6dd80d0be0 con 0x7f6dd80b5770 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6dd759e640 1 --2- 172.21.15.37:0/2915570085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dd80ce700 0x7f6dd8070cf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80739b0 tx=0x7f6dd002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f9eba57dd79cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6dd5d9b640 1 -- 172.21.15.37:0/2915570085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0007740 con 0x7f6dd80ce700 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6dd5d9b640 1 -- 172.21.15.37:0/2915570085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd00078e0 con 0x7f6dd80ce700 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6dd5d9b640 1 -- 172.21.15.37:0/2915570085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0007cb0 con 0x7f6dd80ce700 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6ddd15f640 1 -- 172.21.15.37:0/2915570085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dd80ce700 msgr2=0x7f6dd8070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6ddd15f640 1 --2- 172.21.15.37:0/2915570085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dd80ce700 0x7f6dd8070cf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80739b0 tx=0x7f6dd002a680 comp rx=0 tx=0).stop 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6ddd15f640 1 -- 172.21.15.37:0/2915570085 shutdown_connections 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6ddd15f640 1 --2- 172.21.15.37:0/2915570085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dd80ce700 0x7f6dd8070cf0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6ddd15f640 1 --2- 172.21.15.37:0/2915570085 >> v2:172.21.15.134:3300/0 conn(0x7f6dd815d170 0x7f6dd80cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6ddd15f640 1 -- 172.21.15.37:0/2915570085 >> 172.21.15.37:0/2915570085 conn(0x7f6dd80b4b00 msgr2=0x7f6dd80b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6ddd15f640 1 -- 172.21.15.37:0/2915570085 shutdown_connections 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.531+0000 7f6ddd15f640 1 -- 172.21.15.37:0/2915570085 wait complete. 2024-04-26T20:21:35.540 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.533+0000 7f6ddd15f640 1 Processor -- start 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.533+0000 7f6ddd15f640 1 -- start start 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.533+0000 7f6ddd15f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dd80b5770 0x7f6dd80c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6ddd15f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6dd80ce700 0x7f6dd806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6ddd15f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dd815d170 0x7f6dd815b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6ddd15f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd815d550 con 0x7f6dd80b5770 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6ddd15f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd80707f0 con 0x7f6dd815d170 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6ddd15f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd815d6d0 con 0x7f6dd80ce700 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dd6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dd80b5770 0x7f6dd80c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dd6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dd80b5770 0x7f6dd80c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56570/0 (socket says 172.21.15.37:56570) 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dd6d9d640 1 -- 172.21.15.37:0/587995081 learned_addr learned my addr 172.21.15.37:0/587995081 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dd759e640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dd815d170 0x7f6dd815b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dd659c640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6dd80ce700 0x7f6dd806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dd6d9d640 1 -- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6dd80ce700 msgr2=0x7f6dd806ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:35.541 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dd6d9d640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6dd80ce700 0x7f6dd806ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dd6d9d640 1 -- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dd815d170 msgr2=0x7f6dd815b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dd6d9d640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dd815d170 0x7f6dd815b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dd6d9d640 1 -- 172.21.15.37:0/587995081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd815bec0 con 0x7f6dd80b5770 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dd6d9d640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dd80b5770 0x7f6dd80c5dd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800d5f0 tx=0x7f6dc800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dbf7fe640 1 -- 172.21.15.37:0/587995081 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc80125f0 con 0x7f6dd80b5770 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dbf7fe640 1 -- 172.21.15.37:0/587995081 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc8019070 con 0x7f6dd80b5770 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6dbf7fe640 1 -- 172.21.15.37:0/587995081 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc80129d0 con 0x7f6dd80b5770 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.534+0000 7f6ddd15f640 1 -- 172.21.15.37:0/587995081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd81606f0 con 0x7f6dd80b5770 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.535+0000 7f6ddd15f640 1 -- 172.21.15.37:0/587995081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd8160410 con 0x7f6dd80b5770 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.535+0000 7f6ddd15f640 1 -- 172.21.15.37:0/587995081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd80767e0 con 0x7f6dd80b5770 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.536+0000 7f6dbf7fe640 1 -- 172.21.15.37:0/587995081 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f6dc8012b70 con 0x7f6dd80b5770 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.536+0000 7f6dbf7fe640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6dc406c860 0x7f6dc406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.536+0000 7f6dbf7fe640 1 -- 172.21.15.37:0/587995081 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f6dc808afd0 con 0x7f6dd80b5770 2024-04-26T20:21:35.542 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.536+0000 7f6dd659c640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6dc406c860 0x7f6dc406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:35.543 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.537+0000 7f6dd659c640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6dc406c860 0x7f6dc406ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80c85a0 tx=0x7f6dcc007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:35.543 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.541+0000 7f6dbf7fe640 1 -- 172.21.15.37:0/587995081 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6dd815bec0 con 0x7f6dd80b5770 2024-04-26T20:21:35.729 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.728+0000 7f6ddd15f640 1 -- 172.21.15.37:0/587995081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6dd8074580 con 0x7f6dd80b5770 2024-04-26T20:21:35.730 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.729+0000 7f6dbf7fe640 1 -- 172.21.15.37:0/587995081 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f6dc8059c10 con 0x7f6dd80b5770 2024-04-26T20:21:35.730 INFO:teuthology.misc.health.smithi037.stdout: 2024-04-26T20:21:35.730 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":12,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:33.340099+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:21:35.732 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 -- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6dc406c860 msgr2=0x7f6dc406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:35.732 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6dc406c860 0x7f6dc406ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80c85a0 tx=0x7f6dcc007d50 comp rx=0 tx=0).stop 2024-04-26T20:21:35.732 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 -- 172.21.15.37:0/587995081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dd80b5770 msgr2=0x7f6dd80c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:35.733 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dd80b5770 0x7f6dd80c5dd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800d5f0 tx=0x7f6dc800dab0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.733 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 -- 172.21.15.37:0/587995081 shutdown_connections 2024-04-26T20:21:35.733 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dd815d170 0x7f6dd815b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.733 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6dc406c860 0x7f6dc406ed20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.733 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6dd80ce700 0x7f6dd806ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.733 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 --2- 172.21.15.37:0/587995081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dd80b5770 0x7f6dd80c5dd0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.733 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 -- 172.21.15.37:0/587995081 >> 172.21.15.37:0/587995081 conn(0x7f6dd80b4b00 msgr2=0x7f6dd80b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:35.733 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 -- 172.21.15.37:0/587995081 shutdown_connections 2024-04-26T20:21:35.733 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:35.731+0000 7f6ddd15f640 1 -- 172.21.15.37:0/587995081 wait complete. 2024-04-26T20:21:35.742 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T20:21:42.549 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T20:21:42.654 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.653+0000 7f785df40640 1 Processor -- start 2024-04-26T20:21:42.654 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.653+0000 7f785df40640 1 -- start start 2024-04-26T20:21:42.657 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.654+0000 7f785df40640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f78580b40e0 0x7f78580d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.657 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.654+0000 7f785df40640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78580d3280 con 0x7f785815f560 2024-04-26T20:21:42.657 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.654+0000 7f785df40640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78580cfd60 con 0x7f78580d3ed0 2024-04-26T20:21:42.657 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.654+0000 7f785df40640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f78580d0000 con 0x7f78580b40e0 2024-04-26T20:21:42.657 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.654+0000 7f78577fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f785815f560 0x7f78580d37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47480/0 (socket says 172.21.15.37:47480) 2024-04-26T20:21:42.657 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.654+0000 7f78577fe640 1 -- 172.21.15.37:0/1546381971 learned_addr learned my addr 172.21.15.37:0/1546381971 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:42.657 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.654+0000 7f7857fff640 1 --2- 172.21.15.37:0/1546381971 >> v2:172.21.15.134:3300/0 conn(0x7f78580b40e0 0x7f78580d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.658 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.654+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1831702695 0 0) 0x7f78580d3280 con 0x7f785815f560 2024-04-26T20:21:42.658 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.654+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7840003520 con 0x7f785815f560 2024-04-26T20:21:42.658 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3766221786 0 0) 0x7f7840003520 con 0x7f785815f560 2024-04-26T20:21:42.658 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 >> v2:172.21.15.134:3300/0 conn(0x7f78580b40e0 msgr2=0x7f78580d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:42.658 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 --2- 172.21.15.37:0/1546381971 >> v2:172.21.15.134:3300/0 conn(0x7f78580b40e0 0x7f78580d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.658 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 >> v1:172.21.15.134:6789/0 conn(0x7f78580d3ed0 legacy=0x7f78580b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:42.658 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78580d02e0 con 0x7f785815f560 2024-04-26T20:21:42.659 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f7848001670 con 0x7f785815f560 2024-04-26T20:21:42.659 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 >> v1:172.21.15.37:6789/0 conn(0x7f785815f560 legacy=0x7f78580d37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:42.659 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 --2- 172.21.15.37:0/1546381971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78400039b0 0x7f7840003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.659 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 --2- 172.21.15.37:0/1546381971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7840004800 0x7f7840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.659 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 --2- 172.21.15.37:0/1546381971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78400051c0 0x7f784000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.660 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f78580d0000 con 0x7f7840004800 2024-04-26T20:21:42.660 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f784000c060 con 0x7f78400039b0 2024-04-26T20:21:42.660 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f784000c260 con 0x7f78400051c0 2024-04-26T20:21:42.660 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f7856ffd640 1 --2- 172.21.15.37:0/1546381971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7840004800 0x7f7840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.660 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f78577fe640 1 --2- 172.21.15.37:0/1546381971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78400039b0 0x7f7840003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.660 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f7857fff640 1 --2- 172.21.15.37:0/1546381971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78400051c0 0x7f784000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.661 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f7856ffd640 1 -- 172.21.15.37:0/1546381971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78400051c0 msgr2=0x7f784000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:42.661 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f7856ffd640 1 --2- 172.21.15.37:0/1546381971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78400051c0 0x7f784000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.661 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f7856ffd640 1 -- 172.21.15.37:0/1546381971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78400039b0 msgr2=0x7f7840003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:42.661 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f7856ffd640 1 --2- 172.21.15.37:0/1546381971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78400039b0 0x7f7840003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.661 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f7857fff640 1 -- 172.21.15.37:0/1546381971 reap_dead start 2024-04-26T20:21:42.661 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.655+0000 7f7856ffd640 1 -- 172.21.15.37:0/1546381971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7840003520 con 0x7f7840004800 2024-04-26T20:21:42.662 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.656+0000 7f7856ffd640 1 --2- 172.21.15.37:0/1546381971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7840004800 0x7f7840004c80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f784c001750 tx=0x7f7840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead7b178422ec00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:42.662 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.656+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c001370 con 0x7f7840004800 2024-04-26T20:21:42.662 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.656+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784c0052d0 con 0x7f7840004800 2024-04-26T20:21:42.662 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.656+0000 7f78567fc640 1 -- 172.21.15.37:0/1546381971 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c02abc0 con 0x7f7840004800 2024-04-26T20:21:42.668 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.657+0000 7f785df40640 1 -- 172.21.15.37:0/1546381971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7840004800 msgr2=0x7f7840004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:42.668 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.657+0000 7f785df40640 1 --2- 172.21.15.37:0/1546381971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7840004800 0x7f7840004c80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f784c001750 tx=0x7f7840005830 comp rx=0 tx=0).stop 2024-04-26T20:21:42.668 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.657+0000 7f785df40640 1 -- 172.21.15.37:0/1546381971 shutdown_connections 2024-04-26T20:21:42.669 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.657+0000 7f785df40640 1 --2- 172.21.15.37:0/1546381971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7840004800 0x7f7840004c80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.669 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.657+0000 7f785df40640 1 -- 172.21.15.37:0/1546381971 >> 172.21.15.37:0/1546381971 conn(0x7f785809e520 msgr2=0x7f78580d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:42.669 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.657+0000 7f785df40640 1 -- 172.21.15.37:0/1546381971 shutdown_connections 2024-04-26T20:21:42.669 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.657+0000 7f785df40640 1 -- 172.21.15.37:0/1546381971 wait complete. 2024-04-26T20:21:42.669 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.658+0000 7f785df40640 1 Processor -- start 2024-04-26T20:21:42.669 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.658+0000 7f785df40640 1 -- start start 2024-04-26T20:21:42.670 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.659+0000 7f785df40640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7840004800 0x7f78580c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.670 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.659+0000 7f785df40640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78580b5bf0 0x7f78580cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.670 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.659+0000 7f785df40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78580b6070 0x7f78580cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.670 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.659+0000 7f785df40640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f785815f940 con 0x7f78580b6070 2024-04-26T20:21:42.670 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.659+0000 7f785df40640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78580d08a0 con 0x7f78580b5bf0 2024-04-26T20:21:42.671 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.659+0000 7f785df40640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f78580d0dc0 con 0x7f7840004800 2024-04-26T20:21:42.671 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.659+0000 7f7857fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78580b6070 0x7f78580cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.671 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.659+0000 7f7857fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78580b6070 0x7f78580cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41274/0 (socket says 172.21.15.37:41274) 2024-04-26T20:21:42.671 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.659+0000 7f7857fff640 1 -- 172.21.15.37:0/3984808419 learned_addr learned my addr 172.21.15.37:0/3984808419 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:42.671 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.659+0000 7f78577fe640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7840004800 0x7f78580c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.671 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.659+0000 7f7856ffd640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78580b5bf0 0x7f78580cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.672 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.660+0000 7f7857fff640 1 -- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7840004800 msgr2=0x7f78580c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:42.672 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.660+0000 7f7857fff640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7840004800 0x7f78580c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.672 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.660+0000 7f7857fff640 1 -- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78580b5bf0 msgr2=0x7f78580cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:42.672 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.660+0000 7f7857fff640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78580b5bf0 0x7f78580cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.672 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.660+0000 7f7857fff640 1 -- 172.21.15.37:0/3984808419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78580c8d70 con 0x7f78580b6070 2024-04-26T20:21:42.672 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.660+0000 7f7857fff640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78580b6070 0x7f78580cb210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f785808f490 tx=0x7f785000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:42.672 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.660+0000 7f783ffff640 1 -- 172.21.15.37:0/3984808419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785000fce0 con 0x7f78580b6070 2024-04-26T20:21:42.672 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.660+0000 7f783ffff640 1 -- 172.21.15.37:0/3984808419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785000fe80 con 0x7f78580b6070 2024-04-26T20:21:42.673 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.660+0000 7f785df40640 1 -- 172.21.15.37:0/3984808419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78580c9050 con 0x7f78580b6070 2024-04-26T20:21:42.673 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.660+0000 7f783ffff640 1 -- 172.21.15.37:0/3984808419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850015670 con 0x7f78580b6070 2024-04-26T20:21:42.673 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.660+0000 7f785df40640 1 -- 172.21.15.37:0/3984808419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78580c9610 con 0x7f78580b6070 2024-04-26T20:21:42.673 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.661+0000 7f785df40640 1 -- 172.21.15.37:0/3984808419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78580767e0 con 0x7f78580b6070 2024-04-26T20:21:42.673 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.662+0000 7f783ffff640 1 -- 172.21.15.37:0/3984808419 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7850015810 con 0x7f78580b6070 2024-04-26T20:21:42.673 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.662+0000 7f783ffff640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f784406c8f0 0x7f784406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.673 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.662+0000 7f783ffff640 1 -- 172.21.15.37:0/3984808419 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f785008c550 con 0x7f78580b6070 2024-04-26T20:21:42.673 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.662+0000 7f78577fe640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f784406c8f0 0x7f784406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.673 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.663+0000 7f78577fe640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f784406c8f0 0x7f784406edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f78480011e0 tx=0x7f7848001170 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:42.674 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.667+0000 7f783ffff640 1 -- 172.21.15.37:0/3984808419 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7850019030 con 0x7f78580b6070 2024-04-26T20:21:42.859 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.858+0000 7f785df40640 1 -- 172.21.15.37:0/3984808419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78580d2d50 con 0x7f78580b6070 2024-04-26T20:21:42.860 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.859+0000 7f783ffff640 1 -- 172.21.15.37:0/3984808419 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f7850010040 con 0x7f78580b6070 2024-04-26T20:21:42.860 INFO:teuthology.misc.health.smithi037.stdout: 2024-04-26T20:21:42.860 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":16,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:39.411264+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:21:42.863 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 -- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f784406c8f0 msgr2=0x7f784406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:42.863 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f784406c8f0 0x7f784406edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f78480011e0 tx=0x7f7848001170 comp rx=0 tx=0).stop 2024-04-26T20:21:42.863 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 -- 172.21.15.37:0/3984808419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78580b6070 msgr2=0x7f78580cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:42.863 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78580b6070 0x7f78580cb210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f785808f490 tx=0x7f785000f1b0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.863 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 -- 172.21.15.37:0/3984808419 shutdown_connections 2024-04-26T20:21:42.863 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78580b6070 0x7f78580cb210 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.863 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78580b5bf0 0x7f78580cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.863 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f784406c8f0 0x7f784406edb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.863 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 --2- 172.21.15.37:0/3984808419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7840004800 0x7f78580c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.863 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 -- 172.21.15.37:0/3984808419 >> 172.21.15.37:0/3984808419 conn(0x7f785809e520 msgr2=0x7f78580d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:42.863 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 -- 172.21.15.37:0/3984808419 shutdown_connections 2024-04-26T20:21:42.864 INFO:teuthology.misc.health.smithi037.stderr:2024-04-26T20:21:42.861+0000 7f785df40640 1 -- 172.21.15.37:0/3984808419 wait complete. 2024-04-26T20:21:42.872 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T20:21:42.872 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T20:21:42.873 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:21:42.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:21:42.875 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:21:42.875 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:21:42.875 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:21:42.877 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:21:42.877 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:21:42.877 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:21:42.880 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:21:42.880 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:21:42.880 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:21:42.882 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:21:42.883 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T20:21:42.883 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:21:42.951 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-26T20:21:42.951 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-26T20:21:42.963 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-04-26T20:21:42.964 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-04-26T20:21:42.964 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-04-26T20:21:42.963+0000 7f788faa4640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-26T20:21:42.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.985+0000 7f35a3b5f640 1 Processor -- start 2024-04-26T20:21:42.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.985+0000 7f35a3b5f640 1 -- start start 2024-04-26T20:21:42.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.986+0000 7f35a3b5f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f359c06d250 0x7f359c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.986+0000 7f35a3b5f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f359c160eb0 0x7f359c161300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.986+0000 7f35a3b5f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f359c073ab0 con 0x7f359c159940 2024-04-26T20:21:42.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.986+0000 7f35a3b5f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f359c0aec10 con 0x7f359c06d250 2024-04-26T20:21:42.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.986+0000 7f35a3b5f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f359c06dbd0 con 0x7f359c160eb0 2024-04-26T20:21:42.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.987+0000 7f35a10fa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f359c06d250 0x7f359c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.987+0000 7f35a10fa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f359c06d250 0x7f359c06d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41304/0 (socket says 172.21.15.37:41304) 2024-04-26T20:21:42.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.987+0000 7f35a10fa640 1 -- 172.21.15.37:0/2082566137 learned_addr learned my addr 172.21.15.37:0/2082566137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.987+0000 7f35a20fc640 1 --2- 172.21.15.37:0/2082566137 >> v2:172.21.15.134:3300/0 conn(0x7f359c160eb0 0x7f359c161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.987+0000 7f35a10fa640 1 -- 172.21.15.37:0/2082566137 >> v2:172.21.15.134:3300/0 conn(0x7f359c160eb0 msgr2=0x7f359c161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.987+0000 7f35a10fa640 1 --2- 172.21.15.37:0/2082566137 >> v2:172.21.15.134:3300/0 conn(0x7f359c160eb0 0x7f359c161300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.987+0000 7f35a10fa640 1 -- 172.21.15.37:0/2082566137 >> v1:172.21.15.134:6789/0 conn(0x7f359c159940 legacy=0x7f359c0733a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.987+0000 7f35a10fa640 1 -- 172.21.15.37:0/2082566137 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c15fd60 con 0x7f359c06d250 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.987+0000 7f35a20fc640 1 --2- 172.21.15.37:0/2082566137 >> v2:172.21.15.134:3300/0 conn(0x7f359c160eb0 0x7f359c161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.987+0000 7f35a10fa640 1 --2- 172.21.15.37:0/2082566137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c06d250 0x7f359c06d690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f359c071410 tx=0x7f358c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6894ecd6703fed06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.988+0000 7f35a08f9640 1 -- 172.21.15.37:0/2082566137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c037070 con 0x7f359c06d250 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.988+0000 7f35a08f9640 1 -- 172.21.15.37:0/2082566137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f358c032440 con 0x7f359c06d250 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.988+0000 7f35a08f9640 1 -- 172.21.15.37:0/2082566137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c032810 con 0x7f359c06d250 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.988+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2082566137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c06d250 msgr2=0x7f359c06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.988+0000 7f35a3b5f640 1 --2- 172.21.15.37:0/2082566137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c06d250 0x7f359c06d690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f359c071410 tx=0x7f358c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.988+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2082566137 shutdown_connections 2024-04-26T20:21:42.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.988+0000 7f35a3b5f640 1 --2- 172.21.15.37:0/2082566137 >> v2:172.21.15.134:3300/0 conn(0x7f359c160eb0 0x7f359c161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.988+0000 7f35a3b5f640 1 --2- 172.21.15.37:0/2082566137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c06d250 0x7f359c06d690 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.988+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2082566137 >> 172.21.15.37:0/2082566137 conn(0x7f359c072880 msgr2=0x7f359c072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:42.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.988+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2082566137 shutdown_connections 2024-04-26T20:21:42.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.988+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2082566137 wait complete. 2024-04-26T20:21:42.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.990+0000 7f35a3b5f640 1 Processor -- start 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.990+0000 7f35a3b5f640 1 -- start start 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.990+0000 7f35a3b5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c06d250 0x7f359c1490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.990+0000 7f35a3b5f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f359c159940 0x7f359c15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.990+0000 7f35a3b5f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c160eb0 0x7f359c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.990+0000 7f35a18fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c06d250 0x7f359c1490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.990+0000 7f35a3b5f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f359c160600 con 0x7f359c06d250 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.990+0000 7f35a18fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c06d250 0x7f359c1490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41318/0 (socket says 172.21.15.37:41318) 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.990+0000 7f35a3b5f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f359c160320 con 0x7f359c159940 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.990+0000 7f35a18fb640 1 -- 172.21.15.37:0/2924606875 learned_addr learned my addr 172.21.15.37:0/2924606875 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.990+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2924606875 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f359c1608e0 con 0x7f359c160eb0 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a10fa640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f359c159940 0x7f359c15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a20fc640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c160eb0 0x7f359c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a18fb640 1 -- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c160eb0 msgr2=0x7f359c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a18fb640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c160eb0 0x7f359c15d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a18fb640 1 -- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f359c159940 msgr2=0x7f359c15c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a18fb640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f359c159940 0x7f359c15c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a18fb640 1 -- 172.21.15.37:0/2924606875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c15b5c0 con 0x7f359c06d250 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a10fa640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f359c159940 0x7f359c15c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a20fc640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c160eb0 0x7f359c15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a18fb640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c06d250 0x7f359c1490b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f359800c460 tx=0x7f359800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35927fc640 1 -- 172.21.15.37:0/2924606875 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598016020 con 0x7f359c06d250 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35927fc640 1 -- 172.21.15.37:0/2924606875 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3598012870 con 0x7f359c06d250 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35927fc640 1 -- 172.21.15.37:0/2924606875 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598012c60 con 0x7f359c06d250 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2924606875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c15b8a0 con 0x7f359c06d250 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.991+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2924606875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c15be60 con 0x7f359c06d250 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.992+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2924606875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359c0693e0 con 0x7f359c06d250 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.993+0000 7f35927fc640 1 -- 172.21.15.37:0/2924606875 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3598019020 con 0x7f359c06d250 2024-04-26T20:21:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.993+0000 7f35927fc640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f358006c8f0 0x7f358006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.993+0000 7f35927fc640 1 -- 172.21.15.37:0/2924606875 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f359808b4e0 con 0x7f359c06d250 2024-04-26T20:21:42.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.994+0000 7f35a10fa640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f358006c8f0 0x7f358006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.994+0000 7f35a10fa640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f358006c8f0 0x7f358006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f358c034000 tx=0x7f358c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:42.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:42.998+0000 7f35927fc640 1 -- 172.21.15.37:0/2924606875 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3598020030 con 0x7f359c06d250 2024-04-26T20:21:43.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.184+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2924606875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f359c068630 con 0x7f359c06d250 2024-04-26T20:21:43.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.185+0000 7f35927fc640 1 -- 172.21.15.37:0/2924606875 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f3598059c60 con 0x7f359c06d250 2024-04-26T20:21:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:43.186 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":16,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:39.411264+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:21:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f358006c8f0 msgr2=0x7f358006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f358006c8f0 0x7f358006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f358c034000 tx=0x7f358c02e000 comp rx=0 tx=0).stop 2024-04-26T20:21:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2924606875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c06d250 msgr2=0x7f359c1490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c06d250 0x7f359c1490b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f359800c460 tx=0x7f359800c920 comp rx=0 tx=0).stop 2024-04-26T20:21:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2924606875 shutdown_connections 2024-04-26T20:21:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f358006c8f0 0x7f358006edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c160eb0 0x7f359c15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f359c159940 0x7f359c15c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 --2- 172.21.15.37:0/2924606875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c06d250 0x7f359c1490b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2924606875 >> 172.21.15.37:0/2924606875 conn(0x7f359c072880 msgr2=0x7f359c15ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2924606875 shutdown_connections 2024-04-26T20:21:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.187+0000 7f35a3b5f640 1 -- 172.21.15.37:0/2924606875 wait complete. 2024-04-26T20:21:43.199 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:21:43.199 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:21:43.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.302+0000 7ff700771640 1 Processor -- start 2024-04-26T20:21:43.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.302+0000 7ff700771640 1 -- start start 2024-04-26T20:21:43.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff700771640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff6f815cdf0 0x7ff6f80d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff700771640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f815d1d0 0x7ff6f80b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff700771640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6f80d2590 con 0x7ff6f80d32d0 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff700771640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff6f80d4310 con 0x7ff6f815cdf0 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff700771640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f80d2870 con 0x7ff6f815d1d0 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fdd0c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff6f80d32d0 0x7ff6f80d1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47496/0 (socket says 172.21.15.37:47496) 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fdd0c640 1 -- 172.21.15.37:0/190559081 learned_addr learned my addr 172.21.15.37:0/190559081 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fe50d640 1 --2- 172.21.15.37:0/190559081 >> v2:172.21.15.134:3300/0 conn(0x7ff6f815cdf0 0x7ff6f80d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fed0e640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f815d1d0 0x7ff6f80b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3457753009 0 0) 0x7ff6f80d2590 con 0x7ff6f80d32d0 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6e4003520 con 0x7ff6f80d32d0 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 735562155 0 0) 0x7ff6e4003520 con 0x7ff6f80d32d0 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f815d1d0 msgr2=0x7ff6f80b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f815d1d0 0x7ff6f80b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 >> v2:172.21.15.134:3300/0 conn(0x7ff6f815cdf0 msgr2=0x7ff6f80d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 --2- 172.21.15.37:0/190559081 >> v2:172.21.15.134:3300/0 conn(0x7ff6f815cdf0 0x7ff6f80d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f80b50b0 con 0x7ff6f80d32d0 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7ff6e8001670 con 0x7ff6f80d32d0 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fe50d640 1 --2- 172.21.15.37:0/190559081 >> v2:172.21.15.134:3300/0 conn(0x7ff6f815cdf0 0x7ff6f80d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 >> v1:172.21.15.37:6789/0 conn(0x7ff6f80d32d0 legacy=0x7ff6f80d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 0x7ff6e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e4004800 0x7ff6e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f815d1d0 0x7ff6e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f80d2870 con 0x7ff6e40039b0 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e400b9c0 con 0x7ff6e4004800 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.303+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e400bb40 con 0x7ff6f815d1d0 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.304+0000 7ff6fe50d640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 0x7ff6e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.304+0000 7ff6fdd0c640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e4004800 0x7ff6e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.304+0000 7ff6fed0e640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f815d1d0 0x7ff6e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.304+0000 7ff6fe50d640 1 -- 172.21.15.37:0/190559081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f815d1d0 msgr2=0x7ff6e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.304+0000 7ff6fe50d640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f815d1d0 0x7ff6e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.304+0000 7ff6fe50d640 1 -- 172.21.15.37:0/190559081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e4004800 msgr2=0x7ff6e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.304+0000 7ff6fe50d640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e4004800 0x7ff6e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.304+0000 7ff6fe50d640 1 -- 172.21.15.37:0/190559081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e4003520 con 0x7ff6e40039b0 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.304+0000 7ff6fe50d640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 0x7ff6e4003dd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009c90 tx=0x7ff6e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac1f026b9f733a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.305+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4036070 con 0x7ff6e40039b0 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.305+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f4031400 con 0x7ff6e40039b0 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.305+0000 7ff6fd50b640 1 -- 172.21.15.37:0/190559081 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4009860 con 0x7ff6e40039b0 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.305+0000 7ff700771640 1 -- 172.21.15.37:0/190559081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 msgr2=0x7ff6e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.305+0000 7ff700771640 1 --2- 172.21.15.37:0/190559081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 0x7ff6e4003dd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009c90 tx=0x7ff6e4005830 comp rx=0 tx=0).stop 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.305+0000 7ff6fed0e640 1 -- 172.21.15.37:0/190559081 reap_dead start 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.305+0000 7ff700771640 1 -- 172.21.15.37:0/190559081 shutdown_connections 2024-04-26T20:21:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.305+0000 7ff700771640 1 -- 172.21.15.37:0/190559081 >> 172.21.15.37:0/190559081 conn(0x7ff6f808ec30 msgr2=0x7ff6f809e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:43.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.305+0000 7ff700771640 1 -- 172.21.15.37:0/190559081 shutdown_connections 2024-04-26T20:21:43.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.305+0000 7ff700771640 1 -- 172.21.15.37:0/190559081 wait complete. 2024-04-26T20:21:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.306+0000 7ff700771640 1 Processor -- start 2024-04-26T20:21:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.306+0000 7ff700771640 1 -- start start 2024-04-26T20:21:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff700771640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 0x7ff6f80c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff700771640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f81508a0 0x7ff6f80b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff700771640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f80c9890 0x7ff6f80c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff700771640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f80d21e0 con 0x7ff6e40039b0 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff700771640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f815fcc0 con 0x7ff6f80c9890 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff700771640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f80b7b20 con 0x7ff6f81508a0 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff6fe50d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 0x7ff6f80c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff6fe50d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 0x7ff6f80c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41342/0 (socket says 172.21.15.37:41342) 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff6fe50d640 1 -- 172.21.15.37:0/563848687 learned_addr learned my addr 172.21.15.37:0/563848687 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff6fdd0c640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f81508a0 0x7ff6f80b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff6fed0e640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f80c9890 0x7ff6f80c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff6fe50d640 1 -- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f81508a0 msgr2=0x7ff6f80b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.307+0000 7ff6fe50d640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f81508a0 0x7ff6f80b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.308+0000 7ff6fe50d640 1 -- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f80c9890 msgr2=0x7ff6f80c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.308+0000 7ff6fe50d640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f80c9890 0x7ff6f80c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.308+0000 7ff6fe50d640 1 -- 172.21.15.37:0/563848687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f80b6540 con 0x7ff6e40039b0 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.308+0000 7ff6fed0e640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f80c9890 0x7ff6f80c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.308+0000 7ff6fe50d640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 0x7ff6f80c8cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009db0 tx=0x7ff6f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.308+0000 7ff6eeffd640 1 -- 172.21.15.37:0/563848687 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4040020 con 0x7ff6e40039b0 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.308+0000 7ff6eeffd640 1 -- 172.21.15.37:0/563848687 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f403b680 con 0x7ff6e40039b0 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.308+0000 7ff700771640 1 -- 172.21.15.37:0/563848687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f80b6820 con 0x7ff6e40039b0 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.308+0000 7ff6eeffd640 1 -- 172.21.15.37:0/563848687 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4036040 con 0x7ff6e40039b0 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.308+0000 7ff700771640 1 -- 172.21.15.37:0/563848687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f80b6de0 con 0x7ff6e40039b0 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.309+0000 7ff700771640 1 -- 172.21.15.37:0/563848687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f806a360 con 0x7ff6e40039b0 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.309+0000 7ff6eeffd640 1 -- 172.21.15.37:0/563848687 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6f4043020 con 0x7ff6e40039b0 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.310+0000 7ff6eeffd640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff6dc06c8f0 0x7ff6dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.310+0000 7ff6eeffd640 1 -- 172.21.15.37:0/563848687 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff6f40ac170 con 0x7ff6e40039b0 2024-04-26T20:21:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.310+0000 7ff6fdd0c640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff6dc06c8f0 0x7ff6dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.310+0000 7ff6fdd0c640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff6dc06c8f0 0x7ff6dc06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80b3f30 tx=0x7ff6e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.314+0000 7ff6eeffd640 1 -- 172.21.15.37:0/563848687 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff6f404a020 con 0x7ff6e40039b0 2024-04-26T20:21:43.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.499+0000 7ff700771640 1 -- 172.21.15.37:0/563848687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff6f80b4cf0 con 0x7ff6e40039b0 2024-04-26T20:21:43.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.499+0000 7ff6eeffd640 1 -- 172.21.15.37:0/563848687 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7ff6f4031e20 con 0x7ff6e40039b0 2024-04-26T20:21:43.500 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:21:43.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 -- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff6dc06c8f0 msgr2=0x7ff6dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff6dc06c8f0 0x7ff6dc06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80b3f30 tx=0x7ff6e8009040 comp rx=0 tx=0).stop 2024-04-26T20:21:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 -- 172.21.15.37:0/563848687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 msgr2=0x7ff6f80c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 0x7ff6f80c8cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009db0 tx=0x7ff6f4004730 comp rx=0 tx=0).stop 2024-04-26T20:21:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 -- 172.21.15.37:0/563848687 shutdown_connections 2024-04-26T20:21:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f80c9890 0x7ff6f80c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff6dc06c8f0 0x7ff6dc06edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6f81508a0 0x7ff6f80b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 --2- 172.21.15.37:0/563848687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e40039b0 0x7ff6f80c8cf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 -- 172.21.15.37:0/563848687 >> 172.21.15.37:0/563848687 conn(0x7ff6f808ec30 msgr2=0x7ff6f815c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 -- 172.21.15.37:0/563848687 shutdown_connections 2024-04-26T20:21:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.501+0000 7ff700771640 1 -- 172.21.15.37:0/563848687 wait complete. 2024-04-26T20:21:43.513 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:21:43.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.612+0000 7fa7480dc640 1 Processor -- start 2024-04-26T20:21:43.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.612+0000 7fa7480dc640 1 -- start start 2024-04-26T20:21:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.613+0000 7fa7480dc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74015d310 0x7fa7400d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.613+0000 7fa7480dc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa7400c8210 0x7fa7400d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.613+0000 7fa7480dc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7400ced60 con 0x7fa7400ce860 2024-04-26T20:21:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.613+0000 7fa7480dc640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa7400c6ab0 con 0x7fa7400c8210 2024-04-26T20:21:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.613+0000 7fa7480dc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa7400cf040 con 0x7fa74015d310 2024-04-26T20:21:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.613+0000 7fa745677640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa7400ce860 0x7fa74009ba50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47504/0 (socket says 172.21.15.37:47504) 2024-04-26T20:21:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.613+0000 7fa745677640 1 -- 172.21.15.37:0/2866913229 learned_addr learned my addr 172.21.15.37:0/2866913229 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:43.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.614+0000 7fa745e78640 1 --2- 172.21.15.37:0/2866913229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74015d310 0x7fa7400d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.614+0000 7fa746679640 1 --2- 172.21.15.37:0/2866913229 >> v2:172.21.15.134:3300/0 conn(0x7fa7400c8210 0x7fa7400d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.614+0000 7fa744e76640 1 -- 172.21.15.37:0/2866913229 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2791260476 0 0) 0x7fa7400ced60 con 0x7fa7400ce860 2024-04-26T20:21:43.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.614+0000 7fa744e76640 1 -- 172.21.15.37:0/2866913229 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa728003520 con 0x7fa7400ce860 2024-04-26T20:21:43.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.614+0000 7fa745e78640 1 -- 172.21.15.37:0/2866913229 >> v2:172.21.15.134:3300/0 conn(0x7fa7400c8210 msgr2=0x7fa7400d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:43.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.614+0000 7fa745e78640 1 --2- 172.21.15.37:0/2866913229 >> v2:172.21.15.134:3300/0 conn(0x7fa7400c8210 0x7fa7400d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.614+0000 7fa745e78640 1 -- 172.21.15.37:0/2866913229 >> v1:172.21.15.37:6789/0 conn(0x7fa7400ce860 legacy=0x7fa74009ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.614+0000 7fa744e76640 1 -- 172.21.15.37:0/2866913229 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3404259216 0 0) 0x7fa728003520 con 0x7fa7400ce860 2024-04-26T20:21:43.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.614+0000 7fa745e78640 1 -- 172.21.15.37:0/2866913229 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7400cf320 con 0x7fa74015d310 2024-04-26T20:21:43.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.614+0000 7fa746679640 1 --2- 172.21.15.37:0/2866913229 >> v2:172.21.15.134:3300/0 conn(0x7fa7400c8210 0x7fa7400d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.615+0000 7fa745e78640 1 --2- 172.21.15.37:0/2866913229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74015d310 0x7fa7400d1540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa7380097a0 tx=0x7fa73802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6476fee36ed5c78f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.615+0000 7fa744e76640 1 -- 172.21.15.37:0/2866913229 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738037070 con 0x7fa74015d310 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.615+0000 7fa744e76640 1 -- 172.21.15.37:0/2866913229 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73802ab10 con 0x7fa74015d310 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.615+0000 7fa744e76640 1 -- 172.21.15.37:0/2866913229 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa734001670 con 0x7fa74015d310 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.615+0000 7fa7480dc640 1 -- 172.21.15.37:0/2866913229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74015d310 msgr2=0x7fa7400d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.615+0000 7fa7480dc640 1 --2- 172.21.15.37:0/2866913229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74015d310 0x7fa7400d1540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa7380097a0 tx=0x7fa73802a080 comp rx=0 tx=0).stop 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.616+0000 7fa7480dc640 1 -- 172.21.15.37:0/2866913229 shutdown_connections 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.616+0000 7fa7480dc640 1 --2- 172.21.15.37:0/2866913229 >> v2:172.21.15.134:3300/0 conn(0x7fa7400c8210 0x7fa7400d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.616+0000 7fa7480dc640 1 --2- 172.21.15.37:0/2866913229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74015d310 0x7fa7400d1540 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.616+0000 7fa7480dc640 1 -- 172.21.15.37:0/2866913229 >> 172.21.15.37:0/2866913229 conn(0x7fa7400b4a50 msgr2=0x7fa7400b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.616+0000 7fa7480dc640 1 -- 172.21.15.37:0/2866913229 shutdown_connections 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.616+0000 7fa7480dc640 1 -- 172.21.15.37:0/2866913229 wait complete. 2024-04-26T20:21:43.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.617+0000 7fa7480dc640 1 Processor -- start 2024-04-26T20:21:43.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.617+0000 7fa7480dc640 1 -- start start 2024-04-26T20:21:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa7480dc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7400c8210 0x7fa74015c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa7480dc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7400ce860 0x7fa7401f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa7480dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa74015d310 0x7fa74015e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa7480dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa7400b5970 con 0x7fa74015d310 2024-04-26T20:21:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa7480dc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7400b57f0 con 0x7fa7400c8210 2024-04-26T20:21:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa7480dc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa74015d6f0 con 0x7fa7400ce860 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa746679640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa74015d310 0x7fa74015e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa746679640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa74015d310 0x7fa74015e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41346/0 (socket says 172.21.15.37:41346) 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa746679640 1 -- 172.21.15.37:0/1682902878 learned_addr learned my addr 172.21.15.37:0/1682902878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa745e78640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7400c8210 0x7fa74015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa745677640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7400ce860 0x7fa7401f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa746679640 1 -- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7400ce860 msgr2=0x7fa7401f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa746679640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7400ce860 0x7fa7401f6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa746679640 1 -- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7400c8210 msgr2=0x7fa74015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa746679640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7400c8210 0x7fa74015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa746679640 1 -- 172.21.15.37:0/1682902878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74015f8b0 con 0x7fa74015d310 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa745e78640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7400c8210 0x7fa74015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa745677640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7400ce860 0x7fa7401f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa746679640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa74015d310 0x7fa74015e420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa73000efc0 tx=0x7fa730009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa73e7fc640 1 -- 172.21.15.37:0/1682902878 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730018030 con 0x7fa74015d310 2024-04-26T20:21:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa73e7fc640 1 -- 172.21.15.37:0/1682902878 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73000cd80 con 0x7fa74015d310 2024-04-26T20:21:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa7480dc640 1 -- 172.21.15.37:0/1682902878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74015ed30 con 0x7fa74015d310 2024-04-26T20:21:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.618+0000 7fa73e7fc640 1 -- 172.21.15.37:0/1682902878 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730003bb0 con 0x7fa74015d310 2024-04-26T20:21:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.619+0000 7fa7480dc640 1 -- 172.21.15.37:0/1682902878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74015f2f0 con 0x7fa74015d310 2024-04-26T20:21:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.620+0000 7fa7480dc640 1 -- 172.21.15.37:0/1682902878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7400693e0 con 0x7fa74015d310 2024-04-26T20:21:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.620+0000 7fa73e7fc640 1 -- 172.21.15.37:0/1682902878 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa73001b070 con 0x7fa74015d310 2024-04-26T20:21:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.620+0000 7fa73e7fc640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa72c06c920 0x7fa72c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.620+0000 7fa73e7fc640 1 -- 172.21.15.37:0/1682902878 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa73008c210 con 0x7fa74015d310 2024-04-26T20:21:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.621+0000 7fa745e78640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa72c06c920 0x7fa72c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.621+0000 7fa745e78640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa72c06c920 0x7fa72c06ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa738034000 tx=0x7fa73802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:43.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.625+0000 7fa73e7fc640 1 -- 172.21.15.37:0/1682902878 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa730010040 con 0x7fa74015d310 2024-04-26T20:21:43.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.853+0000 7fa7480dc640 1 -- 172.21.15.37:0/1682902878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7400d1ae0 con 0x7fa74015d310 2024-04-26T20:21:43.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.853+0000 7fa73e7fc640 1 -- 172.21.15.37:0/1682902878 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7fa73005a990 con 0x7fa74015d310 2024-04-26T20:21:43.855 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:43.855 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3920647125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3920647125},{"type":"v1","addr":"172.21.15.37:6834","nonce":3920647125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T20:21:43.855 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-04-26T20:21:43.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.855+0000 7fa7480dc640 1 -- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa72c06c920 msgr2=0x7fa72c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.855+0000 7fa7480dc640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa72c06c920 0x7fa72c06ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa738034000 tx=0x7fa73802e000 comp rx=0 tx=0).stop 2024-04-26T20:21:43.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.855+0000 7fa7480dc640 1 -- 172.21.15.37:0/1682902878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa74015d310 msgr2=0x7fa74015e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.855+0000 7fa7480dc640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa74015d310 0x7fa74015e420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa73000efc0 tx=0x7fa730009f90 comp rx=0 tx=0).stop 2024-04-26T20:21:43.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.856+0000 7fa7480dc640 1 -- 172.21.15.37:0/1682902878 shutdown_connections 2024-04-26T20:21:43.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.856+0000 7fa7480dc640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa74015d310 0x7fa74015e420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.856+0000 7fa7480dc640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa72c06c920 0x7fa72c06ede0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.856+0000 7fa7480dc640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7400ce860 0x7fa7401f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.856+0000 7fa7480dc640 1 --2- 172.21.15.37:0/1682902878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7400c8210 0x7fa74015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.856+0000 7fa7480dc640 1 -- 172.21.15.37:0/1682902878 >> 172.21.15.37:0/1682902878 conn(0x7fa7400b4a50 msgr2=0x7fa7400703d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:43.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.856+0000 7fa7480dc640 1 -- 172.21.15.37:0/1682902878 shutdown_connections 2024-04-26T20:21:43.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:43.856+0000 7fa7480dc640 1 -- 172.21.15.37:0/1682902878 wait complete. 2024-04-26T20:21:45.867 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:21:45.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.979+0000 7f4cac34d640 1 Processor -- start 2024-04-26T20:21:45.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.979+0000 7f4cac34d640 1 -- start start 2024-04-26T20:21:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.980+0000 7f4cac34d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4ca406d560 0x7f4ca40b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.980+0000 7f4cac34d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca40d23e0 0x7f4ca415ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.980+0000 7f4cac34d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca40cd650 con 0x7f4ca409ba50 2024-04-26T20:21:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.980+0000 7f4cac34d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca40cd8f0 con 0x7f4ca406d560 2024-04-26T20:21:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.980+0000 7f4cac34d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca40cdbd0 con 0x7f4ca40d23e0 2024-04-26T20:21:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.981+0000 7f4ca98e8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4ca406d560 0x7f4ca40b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:45.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.981+0000 7f4caa8ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca40d23e0 0x7f4ca415ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:45.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.981+0000 7f4ca98e8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4ca406d560 0x7f4ca40b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:42872/0 (socket says 172.21.15.37:42872) 2024-04-26T20:21:45.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.981+0000 7f4caa0e9640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f4ca409ba50 0x7f4ca409be30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:59718/0 (socket says 172.21.15.37:59718) 2024-04-26T20:21:45.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.981+0000 7f4ca98e8640 1 -- 172.21.15.37:0/1017219247 learned_addr learned my addr 172.21.15.37:0/1017219247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:45.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.981+0000 7f4caa8ea640 1 -- 172.21.15.37:0/1017219247 >> v2:172.21.15.134:3300/0 conn(0x7f4ca406d560 msgr2=0x7f4ca40b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:45.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.981+0000 7f4caa8ea640 1 --2- 172.21.15.37:0/1017219247 >> v2:172.21.15.134:3300/0 conn(0x7f4ca406d560 0x7f4ca40b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:45.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.981+0000 7f4caa8ea640 1 -- 172.21.15.37:0/1017219247 >> v1:172.21.15.134:6789/0 conn(0x7f4ca409ba50 legacy=0x7f4ca409be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.981+0000 7f4caa8ea640 1 -- 172.21.15.37:0/1017219247 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca40cdeb0 con 0x7f4ca40d23e0 2024-04-26T20:21:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.981+0000 7f4ca98e8640 1 --2- 172.21.15.37:0/1017219247 >> v2:172.21.15.134:3300/0 conn(0x7f4ca406d560 0x7f4ca40b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.982+0000 7f4caa8ea640 1 --2- 172.21.15.37:0/1017219247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca40d23e0 0x7f4ca415ebd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0097a0 tx=0x7f4c9c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71e9b5670563e1dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.982+0000 7f4ca90e7640 1 -- 172.21.15.37:0/1017219247 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c037070 con 0x7f4ca40d23e0 2024-04-26T20:21:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.982+0000 7f4ca90e7640 1 -- 172.21.15.37:0/1017219247 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9c032440 con 0x7f4ca40d23e0 2024-04-26T20:21:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.982+0000 7f4ca90e7640 1 -- 172.21.15.37:0/1017219247 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c032810 con 0x7f4ca40d23e0 2024-04-26T20:21:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.982+0000 7f4cac34d640 1 -- 172.21.15.37:0/1017219247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca40d23e0 msgr2=0x7f4ca415ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.983+0000 7f4cac34d640 1 --2- 172.21.15.37:0/1017219247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca40d23e0 0x7f4ca415ebd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0097a0 tx=0x7f4c9c02a180 comp rx=0 tx=0).stop 2024-04-26T20:21:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.983+0000 7f4cac34d640 1 -- 172.21.15.37:0/1017219247 shutdown_connections 2024-04-26T20:21:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.983+0000 7f4cac34d640 1 --2- 172.21.15.37:0/1017219247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca40d23e0 0x7f4ca415ebd0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.983+0000 7f4cac34d640 1 --2- 172.21.15.37:0/1017219247 >> v2:172.21.15.134:3300/0 conn(0x7f4ca406d560 0x7f4ca40b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:45.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.983+0000 7f4cac34d640 1 -- 172.21.15.37:0/1017219247 >> 172.21.15.37:0/1017219247 conn(0x7f4ca40b4a50 msgr2=0x7f4ca40b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:45.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.983+0000 7f4cac34d640 1 -- 172.21.15.37:0/1017219247 shutdown_connections 2024-04-26T20:21:45.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.983+0000 7f4cac34d640 1 -- 172.21.15.37:0/1017219247 wait complete. 2024-04-26T20:21:45.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.984+0000 7f4cac34d640 1 Processor -- start 2024-04-26T20:21:45.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.984+0000 7f4cac34d640 1 -- start start 2024-04-26T20:21:45.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4cac34d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca406d560 0x7f4ca41f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4cac34d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ca409ba50 0x7f4ca41f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4cac34d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca40d23e0 0x7f4ca40b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4cac34d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca40d2860 con 0x7f4ca409ba50 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4cac34d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca40b4790 con 0x7f4ca406d560 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4cac34d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca40d29e0 con 0x7f4ca40d23e0 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4ca98e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ca409ba50 0x7f4ca41f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4ca98e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ca409ba50 0x7f4ca41f9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59266/0 (socket says 172.21.15.37:59266) 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4ca98e8640 1 -- 172.21.15.37:0/2778735564 learned_addr learned my addr 172.21.15.37:0/2778735564 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4caa0e9640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca406d560 0x7f4ca41f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4caa8ea640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca40d23e0 0x7f4ca40b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4ca98e8640 1 -- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca40d23e0 msgr2=0x7f4ca40b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4ca98e8640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca40d23e0 0x7f4ca40b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4ca98e8640 1 -- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca406d560 msgr2=0x7f4ca41f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4ca98e8640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca406d560 0x7f4ca41f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4ca98e8640 1 -- 172.21.15.37:0/2778735564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca41d5220 con 0x7f4ca409ba50 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.985+0000 7f4ca98e8640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ca409ba50 0x7f4ca41f9200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400e9b0 tx=0x7f4c9400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.986+0000 7f4c9affd640 1 -- 172.21.15.37:0/2778735564 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94017070 con 0x7f4ca409ba50 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.986+0000 7f4c9affd640 1 -- 172.21.15.37:0/2778735564 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9400ca00 con 0x7f4ca409ba50 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.986+0000 7f4c9affd640 1 -- 172.21.15.37:0/2778735564 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9400cdf0 con 0x7f4ca409ba50 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.986+0000 7f4cac34d640 1 -- 172.21.15.37:0/2778735564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca41d5500 con 0x7f4ca409ba50 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.986+0000 7f4cac34d640 1 -- 172.21.15.37:0/2778735564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca41d5ac0 con 0x7f4ca409ba50 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.987+0000 7f4cac34d640 1 -- 172.21.15.37:0/2778735564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca40693e0 con 0x7f4ca409ba50 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.987+0000 7f4c9affd640 1 -- 172.21.15.37:0/2778735564 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c9401a070 con 0x7f4ca409ba50 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.988+0000 7f4c9affd640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4c8806c920 0x7f4c8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.988+0000 7f4c9affd640 1 -- 172.21.15.37:0/2778735564 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f4c9408ba50 con 0x7f4ca409ba50 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.988+0000 7f4caa0e9640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4c8806c920 0x7f4c8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.988+0000 7f4caa0e9640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4c8806c920 0x7f4c8806ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ca41f8750 tx=0x7f4ca00016d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:45.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:45.992+0000 7f4c9affd640 1 -- 172.21.15.37:0/2778735564 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c94059b70 con 0x7f4ca409ba50 2024-04-26T20:21:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.219+0000 7f4cac34d640 1 -- 172.21.15.37:0/2778735564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ca406db40 con 0x7f4ca409ba50 2024-04-26T20:21:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.219+0000 7f4c9affd640 1 -- 172.21.15.37:0/2778735564 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f4c9405de70 con 0x7f4ca409ba50 2024-04-26T20:21:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:46.221 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2911741012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2911741012},{"type":"v1","addr":"172.21.15.134:6836","nonce":2911741012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1600483467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1600483467},{"type":"v1","addr":"172.21.15.37:6835","nonce":1600483467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2722614558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2722614558},{"type":"v1","addr":"172.21.15.134:6837","nonce":2722614558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3920647125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3920647125},{"type":"v1","addr":"172.21.15.37:6834","nonce":3920647125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T20:21:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-04-26T20:21:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.221+0000 7f4cac34d640 1 -- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4c8806c920 msgr2=0x7f4c8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.221+0000 7f4cac34d640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4c8806c920 0x7f4c8806ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ca41f8750 tx=0x7f4ca00016d0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.222+0000 7f4cac34d640 1 -- 172.21.15.37:0/2778735564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ca409ba50 msgr2=0x7f4ca41f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.222+0000 7f4cac34d640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ca409ba50 0x7f4ca41f9200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400e9b0 tx=0x7f4c9400ee70 comp rx=0 tx=0).stop 2024-04-26T20:21:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.222+0000 7f4cac34d640 1 -- 172.21.15.37:0/2778735564 shutdown_connections 2024-04-26T20:21:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.222+0000 7f4cac34d640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4c8806c920 0x7f4c8806ede0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.222+0000 7f4cac34d640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca40d23e0 0x7f4ca40b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.222+0000 7f4cac34d640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ca409ba50 0x7f4ca41f9200 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.222+0000 7f4cac34d640 1 --2- 172.21.15.37:0/2778735564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca406d560 0x7f4ca41f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.222+0000 7f4cac34d640 1 -- 172.21.15.37:0/2778735564 >> 172.21.15.37:0/2778735564 conn(0x7f4ca40b4a50 msgr2=0x7f4ca4070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.222+0000 7f4cac34d640 1 -- 172.21.15.37:0/2778735564 shutdown_connections 2024-04-26T20:21:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.222+0000 7f4cac34d640 1 -- 172.21.15.37:0/2778735564 wait complete. 2024-04-26T20:21:46.233 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:21:46.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.333+0000 7fa274e08640 1 Processor -- start 2024-04-26T20:21:46.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.333+0000 7fa274e08640 1 -- start start 2024-04-26T20:21:46.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa274e08640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa270162b40 0x7fa2700d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa274e08640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa2700d3820 0x7fa2700d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa274e08640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2700d1850 0x7fa2700d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa274e08640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa2700d41c0 con 0x7fa270162b40 2024-04-26T20:21:46.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa274e08640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa2700d0210 con 0x7fa2700d3820 2024-04-26T20:21:46.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa274e08640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa2700d04f0 con 0x7fa2700d1850 2024-04-26T20:21:46.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa26e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa270162b40 0x7fa2700d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa26e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa270162b40 0x7fa2700d5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59280/0 (socket says 172.21.15.37:59280) 2024-04-26T20:21:46.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa26e59c640 1 -- 172.21.15.37:0/1619929311 learned_addr learned my addr 172.21.15.37:0/1619929311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:46.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa267fff640 1 --2- 172.21.15.37:0/1619929311 >> v2:172.21.15.134:3300/0 conn(0x7fa2700d3820 0x7fa2700d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa26ed9d640 1 --2- 172.21.15.37:0/1619929311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2700d1850 0x7fa2700d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa26e59c640 1 -- 172.21.15.37:0/1619929311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2700d1850 msgr2=0x7fa2700d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:46.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa26e59c640 1 --2- 172.21.15.37:0/1619929311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2700d1850 0x7fa2700d1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa26e59c640 1 -- 172.21.15.37:0/1619929311 >> v2:172.21.15.134:3300/0 conn(0x7fa2700d3820 msgr2=0x7fa2700d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:46.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa26e59c640 1 --2- 172.21.15.37:0/1619929311 >> v2:172.21.15.134:3300/0 conn(0x7fa2700d3820 0x7fa2700d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa26e59c640 1 -- 172.21.15.37:0/1619929311 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2700d07d0 con 0x7fa270162b40 2024-04-26T20:21:46.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.334+0000 7fa267fff640 1 --2- 172.21.15.37:0/1619929311 >> v2:172.21.15.134:3300/0 conn(0x7fa2700d3820 0x7fa2700d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:46.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.335+0000 7fa26e59c640 1 --2- 172.21.15.37:0/1619929311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa270162b40 0x7fa2700d5bd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa27008f8e0 tx=0x7fa26002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6501800376f27b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:46.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.335+0000 7fa26dd9b640 1 -- 172.21.15.37:0/1619929311 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260037070 con 0x7fa270162b40 2024-04-26T20:21:46.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.335+0000 7fa26dd9b640 1 -- 172.21.15.37:0/1619929311 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa260032440 con 0x7fa270162b40 2024-04-26T20:21:46.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.335+0000 7fa26dd9b640 1 -- 172.21.15.37:0/1619929311 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260032810 con 0x7fa270162b40 2024-04-26T20:21:46.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.335+0000 7fa274e08640 1 -- 172.21.15.37:0/1619929311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa270162b40 msgr2=0x7fa2700d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.335+0000 7fa274e08640 1 --2- 172.21.15.37:0/1619929311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa270162b40 0x7fa2700d5bd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa27008f8e0 tx=0x7fa26002a270 comp rx=0 tx=0).stop 2024-04-26T20:21:46.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.336+0000 7fa274e08640 1 -- 172.21.15.37:0/1619929311 shutdown_connections 2024-04-26T20:21:46.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.336+0000 7fa274e08640 1 --2- 172.21.15.37:0/1619929311 >> v2:172.21.15.134:3300/0 conn(0x7fa2700d3820 0x7fa2700d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.336+0000 7fa274e08640 1 --2- 172.21.15.37:0/1619929311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2700d1850 0x7fa2700d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.336+0000 7fa274e08640 1 --2- 172.21.15.37:0/1619929311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa270162b40 0x7fa2700d5bd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.336+0000 7fa274e08640 1 -- 172.21.15.37:0/1619929311 >> 172.21.15.37:0/1619929311 conn(0x7fa27009e550 msgr2=0x7fa2700d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.336+0000 7fa274e08640 1 -- 172.21.15.37:0/1619929311 shutdown_connections 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.336+0000 7fa274e08640 1 -- 172.21.15.37:0/1619929311 wait complete. 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.337+0000 7fa274e08640 1 Processor -- start 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.337+0000 7fa274e08640 1 -- start start 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa274e08640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2700d1850 0x7fa27014a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa274e08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2700d3820 0x7fa2700cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa274e08640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa270162b40 0x7fa270149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa274e08640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2700d6110 con 0x7fa2700d3820 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa274e08640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2700d0d90 con 0x7fa270162b40 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa274e08640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa2700d12b0 con 0x7fa2700d1850 2024-04-26T20:21:46.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa267fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2700d3820 0x7fa2700cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa267fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2700d3820 0x7fa2700cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59296/0 (socket says 172.21.15.37:59296) 2024-04-26T20:21:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa267fff640 1 -- 172.21.15.37:0/2210970936 learned_addr learned my addr 172.21.15.37:0/2210970936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa26ed9d640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa270162b40 0x7fa270149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa26e59c640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2700d1850 0x7fa27014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa267fff640 1 -- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2700d1850 msgr2=0x7fa27014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa267fff640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2700d1850 0x7fa27014a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa267fff640 1 -- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa270162b40 msgr2=0x7fa270149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa267fff640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa270162b40 0x7fa270149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa267fff640 1 -- 172.21.15.37:0/2210970936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2700cd240 con 0x7fa2700d3820 2024-04-26T20:21:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa267fff640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2700d3820 0x7fa2700cc310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa25800c7f0 tx=0x7fa25800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa266ffd640 1 -- 172.21.15.37:0/2210970936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258015070 con 0x7fa2700d3820 2024-04-26T20:21:46.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa266ffd640 1 -- 172.21.15.37:0/2210970936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa258007a30 con 0x7fa2700d3820 2024-04-26T20:21:46.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa274e08640 1 -- 172.21.15.37:0/2210970936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2700ccc80 con 0x7fa2700d3820 2024-04-26T20:21:46.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa266ffd640 1 -- 172.21.15.37:0/2210970936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258007e00 con 0x7fa2700d3820 2024-04-26T20:21:46.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.338+0000 7fa274e08640 1 -- 172.21.15.37:0/2210970936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2700ce170 con 0x7fa2700d3820 2024-04-26T20:21:46.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.339+0000 7fa274e08640 1 -- 172.21.15.37:0/2210970936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27006a360 con 0x7fa2700d3820 2024-04-26T20:21:46.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.340+0000 7fa266ffd640 1 -- 172.21.15.37:0/2210970936 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa258018070 con 0x7fa2700d3820 2024-04-26T20:21:46.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.340+0000 7fa266ffd640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa25006c8f0 0x7fa25006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.341+0000 7fa266ffd640 1 -- 172.21.15.37:0/2210970936 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa25808b680 con 0x7fa2700d3820 2024-04-26T20:21:46.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.341+0000 7fa26e59c640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa25006c8f0 0x7fa25006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.341+0000 7fa26e59c640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa25006c8f0 0x7fa25006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa260034000 tx=0x7fa26002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:46.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.345+0000 7fa266ffd640 1 -- 172.21.15.37:0/2210970936 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa258059840 con 0x7fa2700d3820 2024-04-26T20:21:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.529+0000 7fa274e08640 1 -- 172.21.15.37:0/2210970936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2700ce340 con 0x7fa2700d3820 2024-04-26T20:21:46.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.530+0000 7fa266ffd640 1 -- 172.21.15.37:0/2210970936 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7fa25805db40 con 0x7fa2700d3820 2024-04-26T20:21:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:46.531 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":16,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:39.411264+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:21:46.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 -- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa25006c8f0 msgr2=0x7fa25006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa25006c8f0 0x7fa25006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa260034000 tx=0x7fa26002e000 comp rx=0 tx=0).stop 2024-04-26T20:21:46.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 -- 172.21.15.37:0/2210970936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2700d3820 msgr2=0x7fa2700cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2700d3820 0x7fa2700cc310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa25800c7f0 tx=0x7fa25800cbf0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 -- 172.21.15.37:0/2210970936 shutdown_connections 2024-04-26T20:21:46.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa270162b40 0x7fa270149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2700d3820 0x7fa2700cc310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa25006c8f0 0x7fa25006edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 --2- 172.21.15.37:0/2210970936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2700d1850 0x7fa27014a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 -- 172.21.15.37:0/2210970936 >> 172.21.15.37:0/2210970936 conn(0x7fa27009e550 msgr2=0x7fa2700d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:46.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 -- 172.21.15.37:0/2210970936 shutdown_connections 2024-04-26T20:21:46.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.532+0000 7fa274e08640 1 -- 172.21.15.37:0/2210970936 wait complete. 2024-04-26T20:21:46.544 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:21:46.544 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:21:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.646+0000 7f8672885640 1 Processor -- start 2024-04-26T20:21:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.646+0000 7f8672885640 1 -- start start 2024-04-26T20:21:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.647+0000 7f8672885640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f866c1610c0 0x7f866c0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.647+0000 7f8672885640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866c0d2150 0x7f866c0d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.647+0000 7f8672885640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f866c161510 con 0x7f866c160800 2024-04-26T20:21:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.647+0000 7f8672885640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f866c0d25c0 con 0x7f866c1610c0 2024-04-26T20:21:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.647+0000 7f8672885640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f866c0d0930 con 0x7f866c0d2150 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.647+0000 7f866b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f866c1610c0 0x7f866c0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.647+0000 7f866b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f866c1610c0 0x7f866c0d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59302/0 (socket says 172.21.15.37:59302) 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.647+0000 7f866b7fe640 1 -- 172.21.15.37:0/4262240971 learned_addr learned my addr 172.21.15.37:0/4262240971 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.647+0000 7f8670e22640 1 --2- 172.21.15.37:0/4262240971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866c0d2150 0x7f866c0d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.648+0000 7f866b7fe640 1 -- 172.21.15.37:0/4262240971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866c0d2150 msgr2=0x7f866c0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.648+0000 7f866b7fe640 1 --2- 172.21.15.37:0/4262240971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866c0d2150 0x7f866c0d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.648+0000 7f866b7fe640 1 -- 172.21.15.37:0/4262240971 >> v1:172.21.15.134:6789/0 conn(0x7f866c160800 legacy=0x7f866c0d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.648+0000 7f866b7fe640 1 -- 172.21.15.37:0/4262240971 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866c0d0c00 con 0x7f866c1610c0 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.648+0000 7f8670e22640 1 --2- 172.21.15.37:0/4262240971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866c0d2150 0x7f866c0d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.648+0000 7f866b7fe640 1 --2- 172.21.15.37:0/4262240971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f866c1610c0 0x7f866c0d1c10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f865c009870 tx=0x7f865c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532ce38b836b84bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.648+0000 7f866affd640 1 -- 172.21.15.37:0/4262240971 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c037070 con 0x7f866c1610c0 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.648+0000 7f866affd640 1 -- 172.21.15.37:0/4262240971 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865c032440 con 0x7f866c1610c0 2024-04-26T20:21:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.648+0000 7f866affd640 1 -- 172.21.15.37:0/4262240971 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c032810 con 0x7f866c1610c0 2024-04-26T20:21:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.649+0000 7f8672885640 1 -- 172.21.15.37:0/4262240971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f866c1610c0 msgr2=0x7f866c0d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.649+0000 7f8672885640 1 --2- 172.21.15.37:0/4262240971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f866c1610c0 0x7f866c0d1c10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f865c009870 tx=0x7f865c02a210 comp rx=0 tx=0).stop 2024-04-26T20:21:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.649+0000 7f8672885640 1 -- 172.21.15.37:0/4262240971 shutdown_connections 2024-04-26T20:21:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.649+0000 7f8672885640 1 --2- 172.21.15.37:0/4262240971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866c0d2150 0x7f866c0d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.649+0000 7f8672885640 1 --2- 172.21.15.37:0/4262240971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f866c1610c0 0x7f866c0d1c10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.649+0000 7f8672885640 1 -- 172.21.15.37:0/4262240971 >> 172.21.15.37:0/4262240971 conn(0x7f866c09e550 msgr2=0x7f866c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.649+0000 7f8672885640 1 -- 172.21.15.37:0/4262240971 shutdown_connections 2024-04-26T20:21:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.649+0000 7f8672885640 1 -- 172.21.15.37:0/4262240971 wait complete. 2024-04-26T20:21:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.650+0000 7f8672885640 1 Processor -- start 2024-04-26T20:21:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.650+0000 7f8672885640 1 -- start start 2024-04-26T20:21:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.651+0000 7f8672885640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f866c0d2150 0x7f866c1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.651+0000 7f8672885640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866c160800 0x7f866c159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.651+0000 7f8672885640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f866c1610c0 0x7f866c159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.651+0000 7f8672885640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f866c0d2e10 con 0x7f866c0d2150 2024-04-26T20:21:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.651+0000 7f8672885640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f866c0d11c0 con 0x7f866c1610c0 2024-04-26T20:21:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.651+0000 7f8672885640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f866c0d30f0 con 0x7f866c160800 2024-04-26T20:21:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.651+0000 7f866bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f866c0d2150 0x7f866c1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.651+0000 7f866bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f866c0d2150 0x7f866c1575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59312/0 (socket says 172.21.15.37:59312) 2024-04-26T20:21:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.651+0000 7f866bfff640 1 -- 172.21.15.37:0/4028870490 learned_addr learned my addr 172.21.15.37:0/4028870490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.651+0000 7f8670e22640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f866c1610c0 0x7f866c159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.651+0000 7f866b7fe640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866c160800 0x7f866c159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.652+0000 7f866bfff640 1 -- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866c160800 msgr2=0x7f866c159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.652+0000 7f866bfff640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866c160800 0x7f866c159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.652+0000 7f866bfff640 1 -- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f866c1610c0 msgr2=0x7f866c159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.652+0000 7f866bfff640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f866c1610c0 0x7f866c159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.652+0000 7f866bfff640 1 -- 172.21.15.37:0/4028870490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866c0ba1c0 con 0x7f866c0d2150 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.652+0000 7f866bfff640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f866c0d2150 0x7f866c1575d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f866000c430 tx=0x7f866000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.652+0000 7f8668ff9640 1 -- 172.21.15.37:0/4028870490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660011790 con 0x7f866c0d2150 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.652+0000 7f8668ff9640 1 -- 172.21.15.37:0/4028870490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8660017070 con 0x7f866c0d2150 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.652+0000 7f8672885640 1 -- 172.21.15.37:0/4028870490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866c0ba4a0 con 0x7f866c0d2150 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.652+0000 7f8668ff9640 1 -- 172.21.15.37:0/4028870490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660011bb0 con 0x7f866c0d2150 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.652+0000 7f8672885640 1 -- 172.21.15.37:0/4028870490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f866c0baa60 con 0x7f866c0d2150 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.653+0000 7f8672885640 1 -- 172.21.15.37:0/4028870490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f866c06a360 con 0x7f866c0d2150 2024-04-26T20:21:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.653+0000 7f8668ff9640 1 -- 172.21.15.37:0/4028870490 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f866001c460 con 0x7f866c0d2150 2024-04-26T20:21:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.654+0000 7f8668ff9640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f865806c8f0 0x7f865806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.654+0000 7f8668ff9640 1 -- 172.21.15.37:0/4028870490 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f866008b4e0 con 0x7f866c0d2150 2024-04-26T20:21:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.654+0000 7f866b7fe640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f865806c8f0 0x7f865806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.654+0000 7f866b7fe640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f865806c8f0 0x7f865806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f865c034000 tx=0x7f865c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:46.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.658+0000 7f8668ff9640 1 -- 172.21.15.37:0/4028870490 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86600596a0 con 0x7f866c0d2150 2024-04-26T20:21:46.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.841+0000 7f8672885640 1 -- 172.21.15.37:0/4028870490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f866c0d0390 con 0x7f866c0d2150 2024-04-26T20:21:46.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.841+0000 7f8668ff9640 1 -- 172.21.15.37:0/4028870490 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f866005d9a0 con 0x7f866c0d2150 2024-04-26T20:21:46.843 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:21:46.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.843+0000 7f8672885640 1 -- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f865806c8f0 msgr2=0x7f865806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.843+0000 7f8672885640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f865806c8f0 0x7f865806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f865c034000 tx=0x7f865c02e000 comp rx=0 tx=0).stop 2024-04-26T20:21:46.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.843+0000 7f8672885640 1 -- 172.21.15.37:0/4028870490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f866c0d2150 msgr2=0x7f866c1575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.843+0000 7f8672885640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f866c0d2150 0x7f866c1575d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f866000c430 tx=0x7f866000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.844+0000 7f8672885640 1 -- 172.21.15.37:0/4028870490 shutdown_connections 2024-04-26T20:21:46.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.844+0000 7f8672885640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f866c1610c0 0x7f866c159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.844+0000 7f8672885640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f865806c8f0 0x7f865806edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.844+0000 7f8672885640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866c160800 0x7f866c159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.844+0000 7f8672885640 1 --2- 172.21.15.37:0/4028870490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f866c0d2150 0x7f866c1575d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.844+0000 7f8672885640 1 -- 172.21.15.37:0/4028870490 >> 172.21.15.37:0/4028870490 conn(0x7f866c09e550 msgr2=0x7f866c15ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:46.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.844+0000 7f8672885640 1 -- 172.21.15.37:0/4028870490 shutdown_connections 2024-04-26T20:21:46.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.844+0000 7f8672885640 1 -- 172.21.15.37:0/4028870490 wait complete. 2024-04-26T20:21:46.856 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:21:46.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.955+0000 7fcea0cfc640 1 Processor -- start 2024-04-26T20:21:46.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.955+0000 7fcea0cfc640 1 -- start start 2024-04-26T20:21:46.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fcea0cfc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fce9c15d310 0x7fce9c0d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fcea0cfc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 0x7fce9c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fcea0cfc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fce9c0d0d40 0x7fce9c09ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fcea0cfc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fce9c0c6ab0 con 0x7fce9c15d310 2024-04-26T20:21:46.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fcea0cfc640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fce9c0ced60 con 0x7fce9c0d0d40 2024-04-26T20:21:46.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fcea0cfc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fce9c0cf040 con 0x7fce9c0ce860 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce9a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fce9c15d310 0x7fce9c0d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce9a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fce9c15d310 0x7fce9c0d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59320/0 (socket says 172.21.15.37:59320) 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce9a59c640 1 -- 172.21.15.37:0/4279262156 learned_addr learned my addr 172.21.15.37:0/4279262156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce91d9b640 1 --2- 172.21.15.37:0/4279262156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 0x7fce9c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce9ad9d640 1 --2- 172.21.15.37:0/4279262156 >> v2:172.21.15.134:3300/0 conn(0x7fce9c0d0d40 0x7fce9c09ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce9a59c640 1 -- 172.21.15.37:0/4279262156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 msgr2=0x7fce9c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce9a59c640 1 --2- 172.21.15.37:0/4279262156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 0x7fce9c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce9a59c640 1 -- 172.21.15.37:0/4279262156 >> v2:172.21.15.134:3300/0 conn(0x7fce9c0d0d40 msgr2=0x7fce9c09ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce9a59c640 1 --2- 172.21.15.37:0/4279262156 >> v2:172.21.15.134:3300/0 conn(0x7fce9c0d0d40 0x7fce9c09ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce9a59c640 1 -- 172.21.15.37:0/4279262156 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c0cf320 con 0x7fce9c15d310 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce91d9b640 1 --2- 172.21.15.37:0/4279262156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 0x7fce9c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.956+0000 7fce9ad9d640 1 --2- 172.21.15.37:0/4279262156 >> v2:172.21.15.134:3300/0 conn(0x7fce9c0d0d40 0x7fce9c09ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.957+0000 7fce9a59c640 1 --2- 172.21.15.37:0/4279262156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce9c15d310 0x7fce9c0d1540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce9c160ed0 tx=0x7fce8c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea8181e40efa9f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.957+0000 7fce99d9b640 1 -- 172.21.15.37:0/4279262156 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c037070 con 0x7fce9c15d310 2024-04-26T20:21:46.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.957+0000 7fce99d9b640 1 -- 172.21.15.37:0/4279262156 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce8c032440 con 0x7fce9c15d310 2024-04-26T20:21:46.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.957+0000 7fce99d9b640 1 -- 172.21.15.37:0/4279262156 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c032810 con 0x7fce9c15d310 2024-04-26T20:21:46.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.958+0000 7fcea0cfc640 1 -- 172.21.15.37:0/4279262156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce9c15d310 msgr2=0x7fce9c0d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.958+0000 7fcea0cfc640 1 --2- 172.21.15.37:0/4279262156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce9c15d310 0x7fce9c0d1540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce9c160ed0 tx=0x7fce8c02a140 comp rx=0 tx=0).stop 2024-04-26T20:21:46.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.958+0000 7fcea0cfc640 1 -- 172.21.15.37:0/4279262156 shutdown_connections 2024-04-26T20:21:46.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.958+0000 7fcea0cfc640 1 --2- 172.21.15.37:0/4279262156 >> v2:172.21.15.134:3300/0 conn(0x7fce9c0d0d40 0x7fce9c09ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.958+0000 7fcea0cfc640 1 --2- 172.21.15.37:0/4279262156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 0x7fce9c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.958+0000 7fcea0cfc640 1 --2- 172.21.15.37:0/4279262156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce9c15d310 0x7fce9c0d1540 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.958+0000 7fcea0cfc640 1 -- 172.21.15.37:0/4279262156 >> 172.21.15.37:0/4279262156 conn(0x7fce9c0b4a50 msgr2=0x7fce9c0b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:46.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.958+0000 7fcea0cfc640 1 -- 172.21.15.37:0/4279262156 shutdown_connections 2024-04-26T20:21:46.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.958+0000 7fcea0cfc640 1 -- 172.21.15.37:0/4279262156 wait complete. 2024-04-26T20:21:46.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.959+0000 7fcea0cfc640 1 Processor -- start 2024-04-26T20:21:46.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.959+0000 7fcea0cfc640 1 -- start start 2024-04-26T20:21:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fcea0cfc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 0x7fce9c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fcea0cfc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0d0d40 0x7fce9c06e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fcea0cfc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce9c15d310 0x7fce9c06eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fcea0cfc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c0b5970 con 0x7fce9c15d310 2024-04-26T20:21:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fcea0cfc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c0b57f0 con 0x7fce9c0d0d40 2024-04-26T20:21:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fcea0cfc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fce9c15d6f0 con 0x7fce9c0ce860 2024-04-26T20:21:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce9ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce9c15d310 0x7fce9c06eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce9ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce9c15d310 0x7fce9c06eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59336/0 (socket says 172.21.15.37:59336) 2024-04-26T20:21:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce9ad9d640 1 -- 172.21.15.37:0/2402972466 learned_addr learned my addr 172.21.15.37:0/2402972466 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce91d9b640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0d0d40 0x7fce9c06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce9a59c640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 0x7fce9c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce9ad9d640 1 -- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 msgr2=0x7fce9c0cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce9ad9d640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 0x7fce9c0cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce9ad9d640 1 -- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0d0d40 msgr2=0x7fce9c06e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce9ad9d640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0d0d40 0x7fce9c06e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce9ad9d640 1 -- 172.21.15.37:0/2402972466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c15db80 con 0x7fce9c15d310 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce91d9b640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0d0d40 0x7fce9c06e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.960+0000 7fce9a59c640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 0x7fce9c0cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.961+0000 7fce9ad9d640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce9c15d310 0x7fce9c06eb60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce9400cef0 tx=0x7fce9400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.961+0000 7fce937fe640 1 -- 172.21.15.37:0/2402972466 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9401a070 con 0x7fce9c15d310 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.961+0000 7fce937fe640 1 -- 172.21.15.37:0/2402972466 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce9400d860 con 0x7fce9c15d310 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.961+0000 7fcea0cfc640 1 -- 172.21.15.37:0/2402972466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9c15de60 con 0x7fce9c15d310 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.961+0000 7fce937fe640 1 -- 172.21.15.37:0/2402972466 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9400dc50 con 0x7fce9c15d310 2024-04-26T20:21:46.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.961+0000 7fcea0cfc640 1 -- 172.21.15.37:0/2402972466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9c06d350 con 0x7fce9c15d310 2024-04-26T20:21:46.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.962+0000 7fcea0cfc640 1 -- 172.21.15.37:0/2402972466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce9c0693e0 con 0x7fce9c15d310 2024-04-26T20:21:46.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.962+0000 7fce937fe640 1 -- 172.21.15.37:0/2402972466 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce9401d070 con 0x7fce9c15d310 2024-04-26T20:21:46.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.963+0000 7fce937fe640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fce7c06c8f0 0x7fce7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.963+0000 7fce937fe640 1 -- 172.21.15.37:0/2402972466 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fce9408bff0 con 0x7fce9c15d310 2024-04-26T20:21:46.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.963+0000 7fce9a59c640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fce7c06c8f0 0x7fce7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.963+0000 7fce9a59c640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fce7c06c8f0 0x7fce7c06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fce8c034000 tx=0x7fce8c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:46.967+0000 7fce937fe640 1 -- 172.21.15.37:0/2402972466 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce9405a8c0 con 0x7fce9c15d310 2024-04-26T20:21:47.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.157+0000 7fcea0cfc640 1 -- 172.21.15.37:0/2402972466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fce9c0d1320 con 0x7fce9c15d310 2024-04-26T20:21:47.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.157+0000 7fce937fe640 1 -- 172.21.15.37:0/2402972466 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fce9405a1c0 con 0x7fce9c15d310 2024-04-26T20:21:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:47.159 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-26T20:21:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:21:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-26T20:21:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-26T20:21:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:21:47.159 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-26T20:21:47.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.159+0000 7fcea0cfc640 1 -- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fce7c06c8f0 msgr2=0x7fce7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.159+0000 7fcea0cfc640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fce7c06c8f0 0x7fce7c06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fce8c034000 tx=0x7fce8c02e000 comp rx=0 tx=0).stop 2024-04-26T20:21:47.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.159+0000 7fcea0cfc640 1 -- 172.21.15.37:0/2402972466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce9c15d310 msgr2=0x7fce9c06eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.159+0000 7fcea0cfc640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce9c15d310 0x7fce9c06eb60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce9400cef0 tx=0x7fce9400a710 comp rx=0 tx=0).stop 2024-04-26T20:21:47.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.159+0000 7fcea0cfc640 1 -- 172.21.15.37:0/2402972466 shutdown_connections 2024-04-26T20:21:47.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.159+0000 7fcea0cfc640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce9c15d310 0x7fce9c06eb60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.159+0000 7fcea0cfc640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0d0d40 0x7fce9c06e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.159+0000 7fcea0cfc640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fce7c06c8f0 0x7fce7c06edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.159+0000 7fcea0cfc640 1 --2- 172.21.15.37:0/2402972466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce9c0ce860 0x7fce9c0cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.159+0000 7fcea0cfc640 1 -- 172.21.15.37:0/2402972466 >> 172.21.15.37:0/2402972466 conn(0x7fce9c0b4a50 msgr2=0x7fce9c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:47.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.160+0000 7fcea0cfc640 1 -- 172.21.15.37:0/2402972466 shutdown_connections 2024-04-26T20:21:47.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.160+0000 7fcea0cfc640 1 -- 172.21.15.37:0/2402972466 wait complete. 2024-04-26T20:21:47.171 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:21:47.171 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:21:47.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.275+0000 7f4b54ccc640 1 Processor -- start 2024-04-26T20:21:47.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.275+0000 7f4b54ccc640 1 -- start start 2024-04-26T20:21:47.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.276+0000 7f4b54ccc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4b5015f410 0x7f4b500b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.276+0000 7f4b54ccc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 0x7f4b500d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.276+0000 7f4b54ccc640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b500b55a0 con 0x7f4b5015f7f0 2024-04-26T20:21:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.276+0000 7f4b54ccc640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4b500b5880 con 0x7f4b5015f410 2024-04-26T20:21:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.276+0000 7f4b54ccc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4b500b5b60 con 0x7f4b500d3e80 2024-04-26T20:21:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.277+0000 7f4b4ed9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 0x7f4b500d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.277+0000 7f4b4e59c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4b5015f410 0x7f4b500b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.277+0000 7f4b4ed9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 0x7f4b500d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:36450/0 (socket says 172.21.15.37:36450) 2024-04-26T20:21:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.277+0000 7f4b4e59c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4b5015f410 0x7f4b500b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:42942/0 (socket says 172.21.15.37:42942) 2024-04-26T20:21:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.277+0000 7f4b4ed9d640 1 -- 172.21.15.37:0/2960811209 learned_addr learned my addr 172.21.15.37:0/2960811209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.277+0000 7f4b4dd9b640 1 --1- 172.21.15.37:0/2960811209 >> v1:172.21.15.134:6789/0 conn(0x7f4b5015f7f0 0x7f4b500d3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:59738/0 (socket says 172.21.15.37:59738) 2024-04-26T20:21:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.277+0000 7f4b4ed9d640 1 -- 172.21.15.37:0/2960811209 >> v2:172.21.15.134:3300/0 conn(0x7f4b5015f410 msgr2=0x7f4b500b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.277+0000 7f4b4ed9d640 1 --2- 172.21.15.37:0/2960811209 >> v2:172.21.15.134:3300/0 conn(0x7f4b5015f410 0x7f4b500b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.277+0000 7f4b4ed9d640 1 -- 172.21.15.37:0/2960811209 >> v1:172.21.15.134:6789/0 conn(0x7f4b5015f7f0 legacy=0x7f4b500d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.277+0000 7f4b4ed9d640 1 -- 172.21.15.37:0/2960811209 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b500b5e40 con 0x7f4b500d3e80 2024-04-26T20:21:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.277+0000 7f4b4e59c640 1 --2- 172.21.15.37:0/2960811209 >> v2:172.21.15.134:3300/0 conn(0x7f4b5015f410 0x7f4b500b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.278+0000 7f4b4ed9d640 1 --2- 172.21.15.37:0/2960811209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 0x7f4b500d0110 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009e50 tx=0x7f4b4802a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c36b16db49aa053e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.278+0000 7f4b4d59a640 1 -- 172.21.15.37:0/2960811209 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48036070 con 0x7f4b500d3e80 2024-04-26T20:21:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.278+0000 7f4b4d59a640 1 -- 172.21.15.37:0/2960811209 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b48031600 con 0x7f4b500d3e80 2024-04-26T20:21:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.278+0000 7f4b4d59a640 1 -- 172.21.15.37:0/2960811209 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b480319d0 con 0x7f4b500d3e80 2024-04-26T20:21:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.279+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2960811209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 msgr2=0x7f4b500d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.279+0000 7f4b54ccc640 1 --2- 172.21.15.37:0/2960811209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 0x7f4b500d0110 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009e50 tx=0x7f4b4802a7a0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.279+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2960811209 shutdown_connections 2024-04-26T20:21:47.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.279+0000 7f4b54ccc640 1 --2- 172.21.15.37:0/2960811209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 0x7f4b500d0110 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.279+0000 7f4b54ccc640 1 --2- 172.21.15.37:0/2960811209 >> v2:172.21.15.134:3300/0 conn(0x7f4b5015f410 0x7f4b500b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.279+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2960811209 >> 172.21.15.37:0/2960811209 conn(0x7f4b5008ebb0 msgr2=0x7f4b5009e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:47.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.279+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2960811209 shutdown_connections 2024-04-26T20:21:47.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.279+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2960811209 wait complete. 2024-04-26T20:21:47.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.280+0000 7f4b54ccc640 1 Processor -- start 2024-04-26T20:21:47.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.280+0000 7f4b54ccc640 1 -- start start 2024-04-26T20:21:47.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b54ccc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 0x7f4b50158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b54ccc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5015f410 0x7f4b501620a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b54ccc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b501625e0 0x7f4b500b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b54ccc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4b500b5240 con 0x7f4b5015f410 2024-04-26T20:21:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b54ccc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b500b50c0 con 0x7f4b501625e0 2024-04-26T20:21:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b54ccc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4b500d0650 con 0x7f4b500d3e80 2024-04-26T20:21:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b4dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5015f410 0x7f4b501620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b4dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5015f410 0x7f4b501620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59338/0 (socket says 172.21.15.37:59338) 2024-04-26T20:21:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b4dd9b640 1 -- 172.21.15.37:0/2675493210 learned_addr learned my addr 172.21.15.37:0/2675493210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b4e59c640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 0x7f4b50158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b4ed9d640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b501625e0 0x7f4b500b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b4dd9b640 1 -- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 msgr2=0x7f4b50158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b4dd9b640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 0x7f4b50158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b4dd9b640 1 -- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b501625e0 msgr2=0x7f4b500b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b4dd9b640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b501625e0 0x7f4b500b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b4dd9b640 1 -- 172.21.15.37:0/2675493210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b500b41e0 con 0x7f4b5015f410 2024-04-26T20:21:47.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b4dd9b640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5015f410 0x7f4b501620a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400c3f0 tx=0x7f4b4400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:47.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b3effd640 1 -- 172.21.15.37:0/2675493210 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440074e0 con 0x7f4b5015f410 2024-04-26T20:21:47.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b3effd640 1 -- 172.21.15.37:0/2675493210 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b44007680 con 0x7f4b5015f410 2024-04-26T20:21:47.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2675493210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b501f62d0 con 0x7f4b5015f410 2024-04-26T20:21:47.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.281+0000 7f4b3effd640 1 -- 172.21.15.37:0/2675493210 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44004a00 con 0x7f4b5015f410 2024-04-26T20:21:47.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.282+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2675493210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b501f6890 con 0x7f4b5015f410 2024-04-26T20:21:47.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.282+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2675493210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b501f6a90 con 0x7f4b5015f410 2024-04-26T20:21:47.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.283+0000 7f4b3effd640 1 -- 172.21.15.37:0/2675493210 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b4401a020 con 0x7f4b5015f410 2024-04-26T20:21:47.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.283+0000 7f4b3effd640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b3006c8f0 0x7f4b3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.283+0000 7f4b3effd640 1 -- 172.21.15.37:0/2675493210 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f4b4408aff0 con 0x7f4b5015f410 2024-04-26T20:21:47.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.284+0000 7f4b4e59c640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b3006c8f0 0x7f4b3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.284+0000 7f4b4e59c640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b3006c8f0 0x7f4b3006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009560 tx=0x7f4b40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:47.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.288+0000 7f4b3effd640 1 -- 172.21.15.37:0/2675493210 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b501f6a90 con 0x7f4b5015f410 2024-04-26T20:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.472+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2675493210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4b500693e0 con 0x7f4b5015f410 2024-04-26T20:21:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.854+0000 7f4b3effd640 1 -- 172.21.15.37:0/2675493210 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f4b44010300 con 0x7f4b5015f410 2024-04-26T20:21:47.856 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-26T20:21:47.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.857+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b3006c8f0 msgr2=0x7f4b3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.857+0000 7f4b54ccc640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b3006c8f0 0x7f4b3006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009560 tx=0x7f4b40008040 comp rx=0 tx=0).stop 2024-04-26T20:21:47.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.857+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2675493210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5015f410 msgr2=0x7f4b501620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.857+0000 7f4b54ccc640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5015f410 0x7f4b501620a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400c3f0 tx=0x7f4b4400c8b0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.857+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2675493210 shutdown_connections 2024-04-26T20:21:47.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.857+0000 7f4b54ccc640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b501625e0 0x7f4b500b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.857+0000 7f4b54ccc640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5015f410 0x7f4b501620a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.858+0000 7f4b54ccc640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b3006c8f0 0x7f4b3006edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.858+0000 7f4b54ccc640 1 --2- 172.21.15.37:0/2675493210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b500d3e80 0x7f4b50158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.858+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2675493210 >> 172.21.15.37:0/2675493210 conn(0x7f4b5008ebb0 msgr2=0x7f4b50075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:47.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.858+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2675493210 shutdown_connections 2024-04-26T20:21:47.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.858+0000 7f4b54ccc640 1 -- 172.21.15.37:0/2675493210 wait complete. 2024-04-26T20:21:47.874 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:21:47.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.983+0000 7f46391cf640 1 Processor -- start 2024-04-26T20:21:47.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.983+0000 7f46391cf640 1 -- start start 2024-04-26T20:21:47.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.984+0000 7f46391cf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f463415f630 0x7f46340d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.984+0000 7f46391cf640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f46340d5ca0 0x7f46340d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.984+0000 7f46391cf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46340d20b0 0x7f46341631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.984+0000 7f46391cf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f46340d0610 con 0x7f46340d20b0 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.984+0000 7f46391cf640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f46340d08f0 con 0x7f46340d5ca0 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.984+0000 7f46391cf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f46340d0bd0 con 0x7f463415f630 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.984+0000 7f463359e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46340d20b0 0x7f46341631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.984+0000 7f463359e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46340d20b0 0x7f46341631f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59348/0 (socket says 172.21.15.37:59348) 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.984+0000 7f463359e640 1 -- 172.21.15.37:0/2799624262 learned_addr learned my addr 172.21.15.37:0/2799624262 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.984+0000 7f463259c640 1 --2- 172.21.15.37:0/2799624262 >> v2:172.21.15.134:3300/0 conn(0x7f46340d5ca0 0x7f46340d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.984+0000 7f4632d9d640 1 --2- 172.21.15.37:0/2799624262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f463415f630 0x7f46340d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.985+0000 7f463359e640 1 -- 172.21.15.37:0/2799624262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f463415f630 msgr2=0x7f46340d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.985+0000 7f463359e640 1 --2- 172.21.15.37:0/2799624262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f463415f630 0x7f46340d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.985+0000 7f463359e640 1 -- 172.21.15.37:0/2799624262 >> v2:172.21.15.134:3300/0 conn(0x7f46340d5ca0 msgr2=0x7f46340d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.985+0000 7f463359e640 1 --2- 172.21.15.37:0/2799624262 >> v2:172.21.15.134:3300/0 conn(0x7f46340d5ca0 0x7f46340d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.985+0000 7f463359e640 1 -- 172.21.15.37:0/2799624262 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46340d0eb0 con 0x7f46340d20b0 2024-04-26T20:21:47.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.985+0000 7f463359e640 1 --2- 172.21.15.37:0/2799624262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46340d20b0 0x7f46341631f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f463409ec60 tx=0x7f462402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdaf27a595ee89cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:47.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.985+0000 7f4631d9b640 1 -- 172.21.15.37:0/2799624262 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624036070 con 0x7f46340d20b0 2024-04-26T20:21:47.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.986+0000 7f4631d9b640 1 -- 172.21.15.37:0/2799624262 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4624031600 con 0x7f46340d20b0 2024-04-26T20:21:47.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.986+0000 7f4631d9b640 1 -- 172.21.15.37:0/2799624262 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46240319d0 con 0x7f46340d20b0 2024-04-26T20:21:47.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.986+0000 7f46391cf640 1 -- 172.21.15.37:0/2799624262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46340d20b0 msgr2=0x7f46341631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.986+0000 7f46391cf640 1 --2- 172.21.15.37:0/2799624262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46340d20b0 0x7f46341631f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f463409ec60 tx=0x7f462402a6a0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.986+0000 7f46391cf640 1 -- 172.21.15.37:0/2799624262 shutdown_connections 2024-04-26T20:21:47.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.986+0000 7f46391cf640 1 --2- 172.21.15.37:0/2799624262 >> v2:172.21.15.134:3300/0 conn(0x7f46340d5ca0 0x7f46340d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.986+0000 7f46391cf640 1 --2- 172.21.15.37:0/2799624262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46340d20b0 0x7f46341631f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.986+0000 7f46391cf640 1 --2- 172.21.15.37:0/2799624262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f463415f630 0x7f46340d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.986+0000 7f46391cf640 1 -- 172.21.15.37:0/2799624262 >> 172.21.15.37:0/2799624262 conn(0x7f463409e530 msgr2=0x7f46340d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.986+0000 7f46391cf640 1 -- 172.21.15.37:0/2799624262 shutdown_connections 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.986+0000 7f46391cf640 1 -- 172.21.15.37:0/2799624262 wait complete. 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.987+0000 7f46391cf640 1 Processor -- start 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.988+0000 7f46391cf640 1 -- start start 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.988+0000 7f46391cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46340d20b0 0x7f46341eae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.988+0000 7f46391cf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46340d5ca0 0x7f463415c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.988+0000 7f46391cf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f463415f630 0x7f46340ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.988+0000 7f46391cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46340d3180 con 0x7f46340d20b0 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.988+0000 7f46391cf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f463415fa10 con 0x7f463415f630 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.988+0000 7f46391cf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f46340d16b0 con 0x7f46340d5ca0 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.988+0000 7f4632d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46340d20b0 0x7f46341eae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.988+0000 7f4632d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46340d20b0 0x7f46341eae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59356/0 (socket says 172.21.15.37:59356) 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.988+0000 7f4632d9d640 1 -- 172.21.15.37:0/529840387 learned_addr learned my addr 172.21.15.37:0/529840387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.988+0000 7f463359e640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f463415f630 0x7f46340ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f463259c640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46340d5ca0 0x7f463415c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f4632d9d640 1 -- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46340d5ca0 msgr2=0x7f463415c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f4632d9d640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46340d5ca0 0x7f463415c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f4632d9d640 1 -- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f463415f630 msgr2=0x7f46340ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f4632d9d640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f463415f630 0x7f46340ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f4632d9d640 1 -- 172.21.15.37:0/529840387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46341e9d70 con 0x7f46340d20b0 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f4632d9d640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46340d20b0 0x7f46341eae10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f462c00e9e0 tx=0x7f462c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f461b7fe640 1 -- 172.21.15.37:0/529840387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c017070 con 0x7f46340d20b0 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f461b7fe640 1 -- 172.21.15.37:0/529840387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462c00c9d0 con 0x7f46340d20b0 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f46391cf640 1 -- 172.21.15.37:0/529840387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46341ea050 con 0x7f46340d20b0 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f461b7fe640 1 -- 172.21.15.37:0/529840387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c00cdc0 con 0x7f46340d20b0 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.989+0000 7f46391cf640 1 -- 172.21.15.37:0/529840387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46341ea610 con 0x7f46340d20b0 2024-04-26T20:21:47.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.990+0000 7f46391cf640 1 -- 172.21.15.37:0/529840387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46340b6700 con 0x7f46340d20b0 2024-04-26T20:21:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.991+0000 7f461b7fe640 1 -- 172.21.15.37:0/529840387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f462c01a070 con 0x7f46340d20b0 2024-04-26T20:21:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.991+0000 7f461b7fe640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f462006c8f0 0x7f462006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.991+0000 7f461b7fe640 1 -- 172.21.15.37:0/529840387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f462c08b9c0 con 0x7f46340d20b0 2024-04-26T20:21:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.991+0000 7f463259c640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f462006c8f0 0x7f462006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.992+0000 7f463259c640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f462006c8f0 0x7f462006edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f46340ca320 tx=0x7f4628009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:47.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:47.996+0000 7f461b7fe640 1 -- 172.21.15.37:0/529840387 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46340b6700 con 0x7f46340d20b0 2024-04-26T20:21:48.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.182+0000 7f46391cf640 1 -- 172.21.15.37:0/529840387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f463406a360 con 0x7f46340d20b0 2024-04-26T20:21:48.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.857+0000 7f461b7fe640 1 -- 172.21.15.37:0/529840387 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f462c08de20 con 0x7f46340d20b0 2024-04-26T20:21:48.859 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-26T20:21:48.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.862+0000 7f46391cf640 1 -- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f462006c8f0 msgr2=0x7f462006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:48.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.862+0000 7f46391cf640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f462006c8f0 0x7f462006edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f46340ca320 tx=0x7f4628009190 comp rx=0 tx=0).stop 2024-04-26T20:21:48.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.862+0000 7f46391cf640 1 -- 172.21.15.37:0/529840387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46340d20b0 msgr2=0x7f46341eae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:48.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.862+0000 7f46391cf640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46340d20b0 0x7f46341eae10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f462c00e9e0 tx=0x7f462c00eea0 comp rx=0 tx=0).stop 2024-04-26T20:21:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.863+0000 7f46391cf640 1 -- 172.21.15.37:0/529840387 shutdown_connections 2024-04-26T20:21:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.863+0000 7f46391cf640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f463415f630 0x7f46340ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.863+0000 7f46391cf640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f462006c8f0 0x7f462006edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.863+0000 7f46391cf640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46340d5ca0 0x7f463415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.863+0000 7f46391cf640 1 --2- 172.21.15.37:0/529840387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46340d20b0 0x7f46341eae10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.863+0000 7f46391cf640 1 -- 172.21.15.37:0/529840387 >> 172.21.15.37:0/529840387 conn(0x7f463409e530 msgr2=0x7f46340b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:48.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.864+0000 7f46391cf640 1 -- 172.21.15.37:0/529840387 shutdown_connections 2024-04-26T20:21:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:48.870+0000 7f46391cf640 1 -- 172.21.15.37:0/529840387 wait complete. 2024-04-26T20:21:48.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:21:49.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.000+0000 7f4196fa5640 1 Processor -- start 2024-04-26T20:21:49.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.000+0000 7f4196fa5640 1 -- start start 2024-04-26T20:21:49.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.000+0000 7f4196fa5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f419015cdf0 0x7f41900d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:49.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.000+0000 7f4196fa5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41900d32d0 0x7f41900d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:49.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.000+0000 7f4196fa5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41900d2590 con 0x7f419015d1d0 2024-04-26T20:21:49.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.000+0000 7f4196fa5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f41900d4310 con 0x7f419015cdf0 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.000+0000 7f4196fa5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f41900d2870 con 0x7f41900d32d0 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.000+0000 7f4195542640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f419015d1d0 0x7f41900b58b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52680/0 (socket says 172.21.15.37:52680) 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.001+0000 7f4195542640 1 -- 172.21.15.37:0/4030078718 learned_addr learned my addr 172.21.15.37:0/4030078718 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.001+0000 7f418ffff640 1 --2- 172.21.15.37:0/4030078718 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41900d32d0 0x7f41900d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.001+0000 7f4194d41640 1 --2- 172.21.15.37:0/4030078718 >> v2:172.21.15.134:3300/0 conn(0x7f419015cdf0 0x7f41900d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.001+0000 7f418f7fe640 1 -- 172.21.15.37:0/4030078718 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 460504802 0 0) 0x7f41900d2590 con 0x7f419015d1d0 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.001+0000 7f418f7fe640 1 -- 172.21.15.37:0/4030078718 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4180003520 con 0x7f419015d1d0 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.001+0000 7f418ffff640 1 -- 172.21.15.37:0/4030078718 >> v2:172.21.15.134:3300/0 conn(0x7f419015cdf0 msgr2=0x7f41900d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.001+0000 7f418ffff640 1 --2- 172.21.15.37:0/4030078718 >> v2:172.21.15.134:3300/0 conn(0x7f419015cdf0 0x7f41900d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.001+0000 7f418ffff640 1 -- 172.21.15.37:0/4030078718 >> v1:172.21.15.37:6789/0 conn(0x7f419015d1d0 legacy=0x7f41900b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.001+0000 7f418f7fe640 1 -- 172.21.15.37:0/4030078718 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3302062832 0 0) 0x7f4180003520 con 0x7f419015d1d0 2024-04-26T20:21:49.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.001+0000 7f418ffff640 1 -- 172.21.15.37:0/4030078718 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41900b50b0 con 0x7f41900d32d0 2024-04-26T20:21:49.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.002+0000 7f418ffff640 1 --2- 172.21.15.37:0/4030078718 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41900d32d0 0x7f41900d1ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f417c0097a0 tx=0x7f417c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d27911e34b29fea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:49.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.002+0000 7f418f7fe640 1 -- 172.21.15.37:0/4030078718 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c037070 con 0x7f41900d32d0 2024-04-26T20:21:49.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.002+0000 7f418f7fe640 1 -- 172.21.15.37:0/4030078718 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417c02aba0 con 0x7f41900d32d0 2024-04-26T20:21:49.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.002+0000 7f418f7fe640 1 -- 172.21.15.37:0/4030078718 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41880019f0 con 0x7f41900d32d0 2024-04-26T20:21:49.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.003+0000 7f4196fa5640 1 -- 172.21.15.37:0/4030078718 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41900d32d0 msgr2=0x7f41900d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:49.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.003+0000 7f4196fa5640 1 --2- 172.21.15.37:0/4030078718 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41900d32d0 0x7f41900d1ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f417c0097a0 tx=0x7f417c02a110 comp rx=0 tx=0).stop 2024-04-26T20:21:49.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.003+0000 7f4196fa5640 1 -- 172.21.15.37:0/4030078718 shutdown_connections 2024-04-26T20:21:49.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.003+0000 7f4196fa5640 1 --2- 172.21.15.37:0/4030078718 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41900d32d0 0x7f41900d1ad0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.003+0000 7f4196fa5640 1 --2- 172.21.15.37:0/4030078718 >> v2:172.21.15.134:3300/0 conn(0x7f419015cdf0 0x7f41900d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.003+0000 7f4196fa5640 1 -- 172.21.15.37:0/4030078718 >> 172.21.15.37:0/4030078718 conn(0x7f419008ec30 msgr2=0x7f419009e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:49.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.003+0000 7f4196fa5640 1 -- 172.21.15.37:0/4030078718 shutdown_connections 2024-04-26T20:21:49.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.003+0000 7f4196fa5640 1 -- 172.21.15.37:0/4030078718 wait complete. 2024-04-26T20:21:49.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.004+0000 7f4196fa5640 1 Processor -- start 2024-04-26T20:21:49.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.004+0000 7f4196fa5640 1 -- start start 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4196fa5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41900d32d0 0x7f41900b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4196fa5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f419015cdf0 0x7f419015c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4196fa5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f419015d1d0 0x7f41900b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4196fa5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41900b5fc0 con 0x7f41900d32d0 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4196fa5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f419015fcc0 con 0x7f419015d1d0 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4196fa5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f41900b6140 con 0x7f419015cdf0 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4194d41640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41900d32d0 0x7f41900b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4194d41640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41900d32d0 0x7f41900b99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59372/0 (socket says 172.21.15.37:59372) 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4194d41640 1 -- 172.21.15.37:0/1973366730 learned_addr learned my addr 172.21.15.37:0/1973366730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4195542640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f419015d1d0 0x7f41900b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f418ffff640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f419015cdf0 0x7f419015c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4194d41640 1 -- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f419015cdf0 msgr2=0x7f419015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4194d41640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f419015cdf0 0x7f419015c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4194d41640 1 -- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f419015d1d0 msgr2=0x7f41900b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4194d41640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f419015d1d0 0x7f41900b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4194d41640 1 -- 172.21.15.37:0/1973366730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41900b8920 con 0x7f41900d32d0 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4195542640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f419015d1d0 0x7f41900b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f418ffff640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f419015cdf0 0x7f419015c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.005+0000 7f4194d41640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41900d32d0 0x7f41900b99c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f418400e9b0 tx=0x7f418400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.006+0000 7f418d7fa640 1 -- 172.21.15.37:0/1973366730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184017070 con 0x7f41900d32d0 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.006+0000 7f418d7fa640 1 -- 172.21.15.37:0/1973366730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f418400c9a0 con 0x7f41900d32d0 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.006+0000 7f4196fa5640 1 -- 172.21.15.37:0/1973366730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41900b8c00 con 0x7f41900d32d0 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.006+0000 7f418d7fa640 1 -- 172.21.15.37:0/1973366730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418400cd90 con 0x7f41900d32d0 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.006+0000 7f4196fa5640 1 -- 172.21.15.37:0/1973366730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41900b91c0 con 0x7f41900d32d0 2024-04-26T20:21:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.007+0000 7f4196fa5640 1 -- 172.21.15.37:0/1973366730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41900b6730 con 0x7f41900d32d0 2024-04-26T20:21:49.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.007+0000 7f418d7fa640 1 -- 172.21.15.37:0/1973366730 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f418401a070 con 0x7f41900d32d0 2024-04-26T20:21:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.008+0000 7f418d7fa640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f417406c8f0 0x7f417406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.008+0000 7f418d7fa640 1 -- 172.21.15.37:0/1973366730 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f418408bbf0 con 0x7f41900d32d0 2024-04-26T20:21:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.008+0000 7f418ffff640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f417406c8f0 0x7f417406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.009+0000 7f418ffff640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f417406c8f0 0x7f417406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f417c034000 tx=0x7f417c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:49.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.012+0000 7f418d7fa640 1 -- 172.21.15.37:0/1973366730 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41900b6730 con 0x7f41900d32d0 2024-04-26T20:21:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.236+0000 7f4196fa5640 1 -- 172.21.15.37:0/1973366730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f419006a360 con 0x7f41900d32d0 2024-04-26T20:21:49.252 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:21:49.254 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:21:49.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.881+0000 7f418d7fa640 1 -- 172.21.15.37:0/1973366730 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f418405a000 con 0x7f41900d32d0 2024-04-26T20:21:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T20:21:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.886+0000 7f4196fa5640 1 -- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f417406c8f0 msgr2=0x7f417406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.887+0000 7f4196fa5640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f417406c8f0 0x7f417406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f417c034000 tx=0x7f417c02e000 comp rx=0 tx=0).stop 2024-04-26T20:21:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.887+0000 7f4196fa5640 1 -- 172.21.15.37:0/1973366730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41900d32d0 msgr2=0x7f41900b99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.887+0000 7f4196fa5640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41900d32d0 0x7f41900b99c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f418400e9b0 tx=0x7f418400ee70 comp rx=0 tx=0).stop 2024-04-26T20:21:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.894+0000 7f4196fa5640 1 -- 172.21.15.37:0/1973366730 shutdown_connections 2024-04-26T20:21:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.894+0000 7f4196fa5640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f419015d1d0 0x7f41900b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.895+0000 7f4196fa5640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f417406c8f0 0x7f417406edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.895+0000 7f4196fa5640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f419015cdf0 0x7f419015c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.895+0000 7f4196fa5640 1 --2- 172.21.15.37:0/1973366730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41900d32d0 0x7f41900b99c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.895+0000 7f4196fa5640 1 -- 172.21.15.37:0/1973366730 >> 172.21.15.37:0/1973366730 conn(0x7f419008ec30 msgr2=0x7f419006e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:49.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.897+0000 7f4196fa5640 1 -- 172.21.15.37:0/1973366730 shutdown_connections 2024-04-26T20:21:49.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:49.898+0000 7f4196fa5640 1 -- 172.21.15.37:0/1973366730 wait complete. 2024-04-26T20:21:49.917 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:21:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.021+0000 7f4bdadda640 1 Processor -- start 2024-04-26T20:21:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.021+0000 7f4bdadda640 1 -- start start 2024-04-26T20:21:50.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.021+0000 7f4bdadda640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4bd406ea40 0x7f4bd40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.021+0000 7f4bdadda640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4bd40cf830 0x7f4bd40cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.021+0000 7f4bdadda640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd40d0180 con 0x7f4bd4162e40 2024-04-26T20:21:50.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.021+0000 7f4bdadda640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd4163220 con 0x7f4bd40cf830 2024-04-26T20:21:50.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.021+0000 7f4bdadda640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd40d0eb0 con 0x7f4bd406ea40 2024-04-26T20:21:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.022+0000 7f4bd9377640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4bd40cf830 0x7f4bd40cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.022+0000 7f4bd8b76640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4bd4162e40 0x7f4bd40b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52690/0 (socket says 172.21.15.37:52690) 2024-04-26T20:21:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.022+0000 7f4bd8b76640 1 -- 172.21.15.37:0/2896513443 learned_addr learned my addr 172.21.15.37:0/2896513443 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.022+0000 7f4bd3fff640 1 --2- 172.21.15.37:0/2896513443 >> v2:172.21.15.134:3300/0 conn(0x7f4bd406ea40 0x7f4bd40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.022+0000 7f4bd9377640 1 -- 172.21.15.37:0/2896513443 >> v2:172.21.15.134:3300/0 conn(0x7f4bd406ea40 msgr2=0x7f4bd40d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.022+0000 7f4bd9377640 1 --2- 172.21.15.37:0/2896513443 >> v2:172.21.15.134:3300/0 conn(0x7f4bd406ea40 0x7f4bd40d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.022+0000 7f4bd9377640 1 -- 172.21.15.37:0/2896513443 >> v1:172.21.15.37:6789/0 conn(0x7f4bd4162e40 legacy=0x7f4bd40b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.022+0000 7f4bd9377640 1 -- 172.21.15.37:0/2896513443 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd40b1d70 con 0x7f4bd40cf830 2024-04-26T20:21:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.022+0000 7f4bd37fe640 1 -- 172.21.15.37:0/2896513443 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2757918873 0 0) 0x7f4bc4001110 con 0x7f4bd4162e40 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bd9377640 1 --2- 172.21.15.37:0/2896513443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd40cf830 0x7f4bd40cfc40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4072bb0 tx=0x7f4bcc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56167f9d65cfdaee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bd37fe640 1 -- 172.21.15.37:0/2896513443 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc007740 con 0x7f4bd40cf830 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bd37fe640 1 -- 172.21.15.37:0/2896513443 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bcc0078e0 con 0x7f4bd40cf830 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bd37fe640 1 -- 172.21.15.37:0/2896513443 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc007cb0 con 0x7f4bd40cf830 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bdadda640 1 -- 172.21.15.37:0/2896513443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd40cf830 msgr2=0x7f4bd40cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bdadda640 1 --2- 172.21.15.37:0/2896513443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd40cf830 0x7f4bd40cfc40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4072bb0 tx=0x7f4bcc02a5f0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bdadda640 1 -- 172.21.15.37:0/2896513443 shutdown_connections 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bdadda640 1 --2- 172.21.15.37:0/2896513443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd40cf830 0x7f4bd40cfc40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bdadda640 1 --2- 172.21.15.37:0/2896513443 >> v2:172.21.15.134:3300/0 conn(0x7f4bd406ea40 0x7f4bd40d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bdadda640 1 -- 172.21.15.37:0/2896513443 >> 172.21.15.37:0/2896513443 conn(0x7f4bd409ba50 msgr2=0x7f4bd409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bdadda640 1 -- 172.21.15.37:0/2896513443 shutdown_connections 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.023+0000 7f4bdadda640 1 -- 172.21.15.37:0/2896513443 wait complete. 2024-04-26T20:21:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.025+0000 7f4bdadda640 1 Processor -- start 2024-04-26T20:21:50.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.025+0000 7f4bdadda640 1 -- start start 2024-04-26T20:21:50.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.025+0000 7f4bdadda640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bd406ea40 0x7f4bd40ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.025+0000 7f4bdadda640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd40cf830 0x7f4bd41f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.025+0000 7f4bdadda640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bd4162e40 0x7f4bd40b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.025+0000 7f4bdadda640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd40d1f20 con 0x7f4bd40cf830 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.025+0000 7f4bdadda640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd40d1c40 con 0x7f4bd406ea40 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.025+0000 7f4bdadda640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd40ce010 con 0x7f4bd4162e40 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd40cf830 0x7f4bd41f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd40cf830 0x7f4bd41f4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59388/0 (socket says 172.21.15.37:59388) 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd3fff640 1 -- 172.21.15.37:0/61421190 learned_addr learned my addr 172.21.15.37:0/61421190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd8b76640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bd406ea40 0x7f4bd40ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd9377640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bd4162e40 0x7f4bd40b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd3fff640 1 -- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bd4162e40 msgr2=0x7f4bd40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd3fff640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bd4162e40 0x7f4bd40b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd3fff640 1 -- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bd406ea40 msgr2=0x7f4bd40ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd3fff640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bd406ea40 0x7f4bd40ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd3fff640 1 -- 172.21.15.37:0/61421190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd40b68c0 con 0x7f4bd40cf830 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd3fff640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd40cf830 0x7f4bd41f4a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00d800 tx=0x7f4bbc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd17fa640 1 -- 172.21.15.37:0/61421190 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00b6c0 con 0x7f4bd40cf830 2024-04-26T20:21:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd17fa640 1 -- 172.21.15.37:0/61421190 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bbc019070 con 0x7f4bd40cf830 2024-04-26T20:21:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bdadda640 1 -- 172.21.15.37:0/61421190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd40b6020 con 0x7f4bd40cf830 2024-04-26T20:21:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bd17fa640 1 -- 172.21.15.37:0/61421190 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00ba80 con 0x7f4bd40cf830 2024-04-26T20:21:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.026+0000 7f4bdadda640 1 -- 172.21.15.37:0/61421190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd40b65e0 con 0x7f4bd40cf830 2024-04-26T20:21:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.027+0000 7f4bdadda640 1 -- 172.21.15.37:0/61421190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd406a360 con 0x7f4bd40cf830 2024-04-26T20:21:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.028+0000 7f4bd17fa640 1 -- 172.21.15.37:0/61421190 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4bbc00bc20 con 0x7f4bd40cf830 2024-04-26T20:21:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.028+0000 7f4bd17fa640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bb406c8f0 0x7f4bb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.029+0000 7f4bd17fa640 1 -- 172.21.15.37:0/61421190 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f4bbc08bed0 con 0x7f4bd40cf830 2024-04-26T20:21:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.029+0000 7f4bd8b76640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bb406c8f0 0x7f4bb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.029+0000 7f4bd8b76640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bb406c8f0 0x7f4bb406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4007fd0 tx=0x7f4bc4007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.033+0000 7f4bd17fa640 1 -- 172.21.15.37:0/61421190 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4bbc059cb0 con 0x7f4bd40cf830 2024-04-26T20:21:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.219+0000 7f4bdadda640 1 -- 172.21.15.37:0/61421190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bd40ce620 con 0x7f4bd40cf830 2024-04-26T20:21:50.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.221+0000 7f4bd17fa640 1 -- 172.21.15.37:0/61421190 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16630 (secure 0 0 0) 0x7f4bbc05dfb0 con 0x7f4bd40cf830 2024-04-26T20:21:50.222 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:50.222 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:49.237892+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.222+0000 7f4bdadda640 1 -- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bb406c8f0 msgr2=0x7f4bb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.222+0000 7f4bdadda640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bb406c8f0 0x7f4bb406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4007fd0 tx=0x7f4bc4007de0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.222+0000 7f4bdadda640 1 -- 172.21.15.37:0/61421190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd40cf830 msgr2=0x7f4bd41f4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.222+0000 7f4bdadda640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd40cf830 0x7f4bd41f4a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00d800 tx=0x7f4bbc00dcc0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.222+0000 7f4bdadda640 1 -- 172.21.15.37:0/61421190 shutdown_connections 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.222+0000 7f4bdadda640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bb406c8f0 0x7f4bb406edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.222+0000 7f4bdadda640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bd4162e40 0x7f4bd40b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.222+0000 7f4bdadda640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd40cf830 0x7f4bd41f4a50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.222+0000 7f4bdadda640 1 --2- 172.21.15.37:0/61421190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bd406ea40 0x7f4bd40ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.223+0000 7f4bdadda640 1 -- 172.21.15.37:0/61421190 >> 172.21.15.37:0/61421190 conn(0x7f4bd409ba50 msgr2=0x7f4bd40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.223+0000 7f4bdadda640 1 -- 172.21.15.37:0/61421190 shutdown_connections 2024-04-26T20:21:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.223+0000 7f4bdadda640 1 -- 172.21.15.37:0/61421190 wait complete. 2024-04-26T20:21:50.235 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:21:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.334+0000 7f3bdf4a9640 1 Processor -- start 2024-04-26T20:21:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.334+0000 7f3bdf4a9640 1 -- start start 2024-04-26T20:21:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.335+0000 7f3bdf4a9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3bd815f380 0x7f3bd80d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.335+0000 7f3bdf4a9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bd80d3de0 con 0x7f3bd815f760 2024-04-26T20:21:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.335+0000 7f3bdf4a9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bd80d40c0 con 0x7f3bd80b9a90 2024-04-26T20:21:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.335+0000 7f3bdf4a9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3bd80d43a0 con 0x7f3bd815f380 2024-04-26T20:21:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.335+0000 7f3bdd245640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3bd815f380 0x7f3bd80d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.335+0000 7f3bdca44640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3bd815f760 0x7f3bd80cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52694/0 (socket says 172.21.15.37:52694) 2024-04-26T20:21:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.335+0000 7f3bdd245640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3bd815f380 0x7f3bd80d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59400/0 (socket says 172.21.15.37:59400) 2024-04-26T20:21:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.335+0000 7f3bdca44640 1 -- 172.21.15.37:0/3511947184 learned_addr learned my addr 172.21.15.37:0/3511947184 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.335+0000 7f3bdd245640 1 -- 172.21.15.37:0/3511947184 >> v1:172.21.15.134:6789/0 conn(0x7f3bd80b9a90 legacy=0x7f3bd80d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.335+0000 7f3bdd245640 1 -- 172.21.15.37:0/3511947184 >> v1:172.21.15.37:6789/0 conn(0x7f3bd815f760 legacy=0x7f3bd80cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.335+0000 7f3bdd245640 1 -- 172.21.15.37:0/3511947184 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd80b8280 con 0x7f3bd815f380 2024-04-26T20:21:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.336+0000 7f3bdd245640 1 --2- 172.21.15.37:0/3511947184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd815f380 0x7f3bd80d1bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40097a0 tx=0x7f3bd402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c737c1271df8271c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.336+0000 7f3bcffff640 1 -- 172.21.15.37:0/3511947184 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4037070 con 0x7f3bd815f380 2024-04-26T20:21:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.336+0000 7f3bcffff640 1 -- 172.21.15.37:0/3511947184 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd4032640 con 0x7f3bd815f380 2024-04-26T20:21:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.336+0000 7f3bcffff640 1 -- 172.21.15.37:0/3511947184 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4032a10 con 0x7f3bd815f380 2024-04-26T20:21:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.336+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/3511947184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd815f380 msgr2=0x7f3bd80d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.336+0000 7f3bdf4a9640 1 --2- 172.21.15.37:0/3511947184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd815f380 0x7f3bd80d1bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40097a0 tx=0x7f3bd402a2c0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.336+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/3511947184 shutdown_connections 2024-04-26T20:21:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.336+0000 7f3bdf4a9640 1 --2- 172.21.15.37:0/3511947184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd815f380 0x7f3bd80d1bd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.336+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/3511947184 >> 172.21.15.37:0/3511947184 conn(0x7f3bd809e560 msgr2=0x7f3bd80d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.336+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/3511947184 shutdown_connections 2024-04-26T20:21:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.336+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/3511947184 wait complete. 2024-04-26T20:21:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.338+0000 7f3bdf4a9640 1 Processor -- start 2024-04-26T20:21:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.338+0000 7f3bdf4a9640 1 -- start start 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.338+0000 7f3bdf4a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd80b9a90 0x7f3bd8154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.338+0000 7f3bdf4a9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bd815f380 0x7f3bd8161dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.338+0000 7f3bdf4a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd815f760 0x7f3bd8162310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.338+0000 7f3bdf4a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd80d1100 con 0x7f3bd80b9a90 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.338+0000 7f3bdf4a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd80d0cc0 con 0x7f3bd815f760 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.338+0000 7f3bdf4a9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd80d1280 con 0x7f3bd815f380 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.338+0000 7f3bdd245640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd80b9a90 0x7f3bd8154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdd245640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd80b9a90 0x7f3bd8154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59416/0 (socket says 172.21.15.37:59416) 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdd245640 1 -- 172.21.15.37:0/2500057102 learned_addr learned my addr 172.21.15.37:0/2500057102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdda46640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd815f760 0x7f3bd8162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdca44640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bd815f380 0x7f3bd8161dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdd245640 1 -- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bd815f380 msgr2=0x7f3bd8161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdd245640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bd815f380 0x7f3bd8161dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdd245640 1 -- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd815f760 msgr2=0x7f3bd8162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdd245640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd815f760 0x7f3bd8162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdd245640 1 -- 172.21.15.37:0/2500057102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd80c9db0 con 0x7f3bd80b9a90 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdda46640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd815f760 0x7f3bd8162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdd245640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd80b9a90 0x7f3bd8154820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3bd402a300 tx=0x7f3bd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bcdffb640 1 -- 172.21.15.37:0/2500057102 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4037040 con 0x7f3bd80b9a90 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bcdffb640 1 -- 172.21.15.37:0/2500057102 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd4042070 con 0x7f3bd80b9a90 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/2500057102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd80ca090 con 0x7f3bd80b9a90 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bcdffb640 1 -- 172.21.15.37:0/2500057102 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4030bb0 con 0x7f3bd80b9a90 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.339+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/2500057102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd80ca650 con 0x7f3bd80b9a90 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.340+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/2500057102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd806a360 con 0x7f3bd80b9a90 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.341+0000 7f3bcdffb640 1 -- 172.21.15.37:0/2500057102 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3bd4047460 con 0x7f3bd80b9a90 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.341+0000 7f3bcdffb640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bbc06c8f0 0x7f3bbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.341+0000 7f3bcdffb640 1 -- 172.21.15.37:0/2500057102 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f3bd40ac620 con 0x7f3bd80b9a90 2024-04-26T20:21:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.341+0000 7f3bdca44640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bbc06c8f0 0x7f3bbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.342+0000 7f3bdca44640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bbc06c8f0 0x7f3bbc06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80c8d50 tx=0x7f3bc8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.345+0000 7f3bcdffb640 1 -- 172.21.15.37:0/2500057102 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3bd407aae0 con 0x7f3bd80b9a90 2024-04-26T20:21:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.531+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/2500057102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bd80b7d30 con 0x7f3bd80b9a90 2024-04-26T20:21:50.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.532+0000 7f3bcdffb640 1 -- 172.21.15.37:0/2500057102 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16630 (secure 0 0 0) 0x7f3bd407a3e0 con 0x7f3bd80b9a90 2024-04-26T20:21:50.534 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:50.534 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:49.237892+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:21:50.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bbc06c8f0 msgr2=0x7f3bbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bbc06c8f0 0x7f3bbc06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80c8d50 tx=0x7f3bc8009040 comp rx=0 tx=0).stop 2024-04-26T20:21:50.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/2500057102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd80b9a90 msgr2=0x7f3bd8154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd80b9a90 0x7f3bd8154820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3bd402a300 tx=0x7f3bd40046c0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/2500057102 shutdown_connections 2024-04-26T20:21:50.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd815f760 0x7f3bd8162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bbc06c8f0 0x7f3bbc06edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bd815f380 0x7f3bd8161dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 --2- 172.21.15.37:0/2500057102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd80b9a90 0x7f3bd8154820 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/2500057102 >> 172.21.15.37:0/2500057102 conn(0x7f3bd809e560 msgr2=0x7f3bd806d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:50.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/2500057102 shutdown_connections 2024-04-26T20:21:50.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.534+0000 7f3bdf4a9640 1 -- 172.21.15.37:0/2500057102 wait complete. 2024-04-26T20:21:50.546 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:21:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.649+0000 7fbe3e3a4640 1 Processor -- start 2024-04-26T20:21:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.649+0000 7fbe3e3a4640 1 -- start start 2024-04-26T20:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe3e3a4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbe381610c0 0x7fbe380d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe3e3a4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbe380d2150 0x7fbe380d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe3e3a4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe38161510 con 0x7fbe38160800 2024-04-26T20:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe3e3a4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fbe380d25c0 con 0x7fbe381610c0 2024-04-26T20:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe3e3a4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fbe380d0930 con 0x7fbe380d2150 2024-04-26T20:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe377fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbe381610c0 0x7fbe380d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe377fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbe381610c0 0x7fbe380d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59432/0 (socket says 172.21.15.37:59432) 2024-04-26T20:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe377fe640 1 -- 172.21.15.37:0/2731903312 learned_addr learned my addr 172.21.15.37:0/2731903312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe3c941640 1 --2- 172.21.15.37:0/2731903312 >> v2:172.21.15.134:3300/0 conn(0x7fbe380d2150 0x7fbe380d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe377fe640 1 -- 172.21.15.37:0/2731903312 >> v2:172.21.15.134:3300/0 conn(0x7fbe380d2150 msgr2=0x7fbe380d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe377fe640 1 --2- 172.21.15.37:0/2731903312 >> v2:172.21.15.134:3300/0 conn(0x7fbe380d2150 0x7fbe380d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe377fe640 1 -- 172.21.15.37:0/2731903312 >> v1:172.21.15.134:6789/0 conn(0x7fbe38160800 legacy=0x7fbe380d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.650+0000 7fbe377fe640 1 -- 172.21.15.37:0/2731903312 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe380d0c00 con 0x7fbe381610c0 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe377fe640 1 --2- 172.21.15.37:0/2731903312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe381610c0 0x7fbe380d1c10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe380d4d40 tx=0x7fbe2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d29c172cc3a73d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe36ffd640 1 -- 172.21.15.37:0/2731903312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c037070 con 0x7fbe381610c0 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe36ffd640 1 -- 172.21.15.37:0/2731903312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe2c032440 con 0x7fbe381610c0 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe36ffd640 1 -- 172.21.15.37:0/2731903312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c032810 con 0x7fbe381610c0 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/2731903312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe381610c0 msgr2=0x7fbe380d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe3e3a4640 1 --2- 172.21.15.37:0/2731903312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe381610c0 0x7fbe380d1c10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe380d4d40 tx=0x7fbe2c02a270 comp rx=0 tx=0).stop 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/2731903312 shutdown_connections 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe3e3a4640 1 --2- 172.21.15.37:0/2731903312 >> v2:172.21.15.134:3300/0 conn(0x7fbe380d2150 0x7fbe380d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe3e3a4640 1 --2- 172.21.15.37:0/2731903312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe381610c0 0x7fbe380d1c10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/2731903312 >> 172.21.15.37:0/2731903312 conn(0x7fbe3809e550 msgr2=0x7fbe380d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/2731903312 shutdown_connections 2024-04-26T20:21:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.651+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/2731903312 wait complete. 2024-04-26T20:21:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.653+0000 7fbe3e3a4640 1 Processor -- start 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.653+0000 7fbe3e3a4640 1 -- start start 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.653+0000 7fbe3e3a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe380d2150 0x7fbe380ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.653+0000 7fbe3e3a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe38160800 0x7fbe380b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.653+0000 7fbe3e3a4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe381610c0 0x7fbe380b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.653+0000 7fbe3e3a4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbe380d2e10 con 0x7fbe38160800 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.653+0000 7fbe3e3a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe380d11c0 con 0x7fbe380d2150 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.653+0000 7fbe3e3a4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbe380d30f0 con 0x7fbe381610c0 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe38160800 0x7fbe380b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe38160800 0x7fbe380b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59442/0 (socket says 172.21.15.37:59442) 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe377fe640 1 -- 172.21.15.37:0/901700136 learned_addr learned my addr 172.21.15.37:0/901700136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe37fff640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe380d2150 0x7fbe380ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe3c941640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe381610c0 0x7fbe380b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe377fe640 1 -- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe381610c0 msgr2=0x7fbe380b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe377fe640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe381610c0 0x7fbe380b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe377fe640 1 -- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe380d2150 msgr2=0x7fbe380ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe377fe640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe380d2150 0x7fbe380ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe377fe640 1 -- 172.21.15.37:0/901700136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe380b5bf0 con 0x7fbe38160800 2024-04-26T20:21:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe377fe640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe38160800 0x7fbe380b4d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c02a2b0 tx=0x7fbe2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe34ff9640 1 -- 172.21.15.37:0/901700136 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c041020 con 0x7fbe38160800 2024-04-26T20:21:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe34ff9640 1 -- 172.21.15.37:0/901700136 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe2c0306c0 con 0x7fbe38160800 2024-04-26T20:21:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe34ff9640 1 -- 172.21.15.37:0/901700136 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c037040 con 0x7fbe38160800 2024-04-26T20:21:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/901700136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe380cd400 con 0x7fbe38160800 2024-04-26T20:21:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.654+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/901700136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe380cd9c0 con 0x7fbe38160800 2024-04-26T20:21:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.655+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/901700136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3806a360 con 0x7fbe38160800 2024-04-26T20:21:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.656+0000 7fbe34ff9640 1 -- 172.21.15.37:0/901700136 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe2c044020 con 0x7fbe38160800 2024-04-26T20:21:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.656+0000 7fbe34ff9640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbe2406c8f0 0x7fbe2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.656+0000 7fbe34ff9640 1 -- 172.21.15.37:0/901700136 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fbe2c0ac2d0 con 0x7fbe38160800 2024-04-26T20:21:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.656+0000 7fbe37fff640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbe2406c8f0 0x7fbe2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.657+0000 7fbe37fff640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbe2406c8f0 0x7fbe2406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe28007fd0 tx=0x7fbe28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.660+0000 7fbe34ff9640 1 -- 172.21.15.37:0/901700136 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbe2c04b030 con 0x7fbe38160800 2024-04-26T20:21:50.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:50.885+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/901700136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbe3815d680 con 0x7fbe38160800 2024-04-26T20:21:51.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.890+0000 7fbe34ff9640 1 -- 172.21.15.37:0/901700136 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7fbe2c07a790 con 0x7fbe38160800 2024-04-26T20:21:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbe2406c8f0 msgr2=0x7fbe2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbe2406c8f0 0x7fbe2406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe28007fd0 tx=0x7fbe28009040 comp rx=0 tx=0).stop 2024-04-26T20:21:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/901700136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe38160800 msgr2=0x7fbe380b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe38160800 0x7fbe380b4d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c02a2b0 tx=0x7fbe2c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:21:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/901700136 shutdown_connections 2024-04-26T20:21:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbe2406c8f0 0x7fbe2406edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe381610c0 0x7fbe380b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:51.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe38160800 0x7fbe380b4d30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:51.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 --2- 172.21.15.37:0/901700136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe380d2150 0x7fbe380ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:51.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/901700136 >> 172.21.15.37:0/901700136 conn(0x7fbe3809e550 msgr2=0x7fbe3806d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:51.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/901700136 shutdown_connections 2024-04-26T20:21:51.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:51.893+0000 7fbe3e3a4640 1 -- 172.21.15.37:0/901700136 wait complete. 2024-04-26T20:21:51.907 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:21:51.907 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:21:52.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.016+0000 7f4b61c20640 1 Processor -- start 2024-04-26T20:21:52.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.016+0000 7f4b61c20640 1 -- start start 2024-04-26T20:21:52.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b61c20640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4b5c15d320 0x7f4b5c0d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:52.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b61c20640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4b5c0b4200 0x7f4b5c0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:52.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b61c20640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5c15d700 con 0x7f4b5c0d1090 2024-04-26T20:21:52.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b61c20640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5c0ceeb0 con 0x7f4b5c0b4200 2024-04-26T20:21:52.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b61c20640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5c0ce8f0 con 0x7f4b5c15d320 2024-04-26T20:21:52.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b5bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4b5c0b4200 0x7f4b5c0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:52.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b5affd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4b5c0d1090 0x7f4b5c0cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52706/0 (socket says 172.21.15.37:52706) 2024-04-26T20:21:52.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b5affd640 1 -- 172.21.15.37:0/3621995956 learned_addr learned my addr 172.21.15.37:0/3621995956 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:52.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b5bfff640 1 -- 172.21.15.37:0/3621995956 >> v2:172.21.15.134:3300/0 conn(0x7f4b5c15d320 msgr2=0x7f4b5c0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:52.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b5b7fe640 1 --2- 172.21.15.37:0/3621995956 >> v2:172.21.15.134:3300/0 conn(0x7f4b5c15d320 0x7f4b5c0d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:52.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b5bfff640 1 --2- 172.21.15.37:0/3621995956 >> v2:172.21.15.134:3300/0 conn(0x7f4b5c15d320 0x7f4b5c0d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b5a7fc640 1 -- 172.21.15.37:0/3621995956 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 716435953 0 0) 0x7f4b5c15d700 con 0x7f4b5c0d1090 2024-04-26T20:21:52.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b5bfff640 1 -- 172.21.15.37:0/3621995956 >> v1:172.21.15.37:6789/0 conn(0x7f4b5c0d1090 legacy=0x7f4b5c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:52.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b5bfff640 1 -- 172.21.15.37:0/3621995956 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c0cebd0 con 0x7f4b5c0b4200 2024-04-26T20:21:52.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.017+0000 7f4b5b7fe640 1 --2- 172.21.15.37:0/3621995956 >> v2:172.21.15.134:3300/0 conn(0x7f4b5c15d320 0x7f4b5c0d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:52.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.018+0000 7f4b5bfff640 1 --2- 172.21.15.37:0/3621995956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c0b4200 0x7f4b5c0d1520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009d90 tx=0x7f4b5402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac2d4edaf4a238f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:52.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.018+0000 7f4b5a7fc640 1 -- 172.21.15.37:0/3621995956 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54031600 con 0x7f4b5c0b4200 2024-04-26T20:21:52.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.018+0000 7f4b5a7fc640 1 -- 172.21.15.37:0/3621995956 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b540317a0 con 0x7f4b5c0b4200 2024-04-26T20:21:52.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.018+0000 7f4b5a7fc640 1 -- 172.21.15.37:0/3621995956 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54031b70 con 0x7f4b5c0b4200 2024-04-26T20:21:52.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.019+0000 7f4b61c20640 1 -- 172.21.15.37:0/3621995956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c0b4200 msgr2=0x7f4b5c0d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:52.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.019+0000 7f4b61c20640 1 --2- 172.21.15.37:0/3621995956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c0b4200 0x7f4b5c0d1520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009d90 tx=0x7f4b5402a610 comp rx=0 tx=0).stop 2024-04-26T20:21:52.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.019+0000 7f4b61c20640 1 -- 172.21.15.37:0/3621995956 shutdown_connections 2024-04-26T20:21:52.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.019+0000 7f4b61c20640 1 --2- 172.21.15.37:0/3621995956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c0b4200 0x7f4b5c0d1520 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.019+0000 7f4b61c20640 1 --2- 172.21.15.37:0/3621995956 >> v2:172.21.15.134:3300/0 conn(0x7f4b5c15d320 0x7f4b5c0d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.019+0000 7f4b61c20640 1 -- 172.21.15.37:0/3621995956 >> 172.21.15.37:0/3621995956 conn(0x7f4b5c0d2f80 msgr2=0x7f4b5c0d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:52.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.019+0000 7f4b61c20640 1 -- 172.21.15.37:0/3621995956 shutdown_connections 2024-04-26T20:21:52.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.019+0000 7f4b61c20640 1 -- 172.21.15.37:0/3621995956 wait complete. 2024-04-26T20:21:52.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.020+0000 7f4b61c20640 1 Processor -- start 2024-04-26T20:21:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.020+0000 7f4b61c20640 1 -- start start 2024-04-26T20:21:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b61c20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c0b4200 0x7f4b5c150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b61c20640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b5c0d1090 0x7f4b5c156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b61c20640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b5c15d320 0x7f4b5c1f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b61c20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c0ce550 con 0x7f4b5c0b4200 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b61c20640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c0b2cd0 con 0x7f4b5c0d1090 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b61c20640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5c0ce6d0 con 0x7f4b5c15d320 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c0b4200 0x7f4b5c150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c0b4200 0x7f4b5c150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59464/0 (socket says 172.21.15.37:59464) 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5b7fe640 1 -- 172.21.15.37:0/596237702 learned_addr learned my addr 172.21.15.37:0/596237702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5affd640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b5c0d1090 0x7f4b5c156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5bfff640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b5c15d320 0x7f4b5c1f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5b7fe640 1 -- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b5c15d320 msgr2=0x7f4b5c1f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5b7fe640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b5c15d320 0x7f4b5c1f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5b7fe640 1 -- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b5c0d1090 msgr2=0x7f4b5c156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5b7fe640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b5c0d1090 0x7f4b5c156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5b7fe640 1 -- 172.21.15.37:0/596237702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c1f75e0 con 0x7f4b5c0b4200 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5affd640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b5c0d1090 0x7f4b5c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.021+0000 7f4b5bfff640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b5c15d320 0x7f4b5c1f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.022+0000 7f4b5b7fe640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c0b4200 0x7f4b5c150fb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c7f0 tx=0x7f4b4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.022+0000 7f4b43fff640 1 -- 172.21.15.37:0/596237702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c015070 con 0x7f4b5c0b4200 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.022+0000 7f4b43fff640 1 -- 172.21.15.37:0/596237702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4c007a30 con 0x7f4b5c0b4200 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.022+0000 7f4b61c20640 1 -- 172.21.15.37:0/596237702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c1f3900 con 0x7f4b5c0b4200 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.022+0000 7f4b43fff640 1 -- 172.21.15.37:0/596237702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c007e00 con 0x7f4b5c0b4200 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.022+0000 7f4b61c20640 1 -- 172.21.15.37:0/596237702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c1f3ec0 con 0x7f4b5c0b4200 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.023+0000 7f4b61c20640 1 -- 172.21.15.37:0/596237702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c0693e0 con 0x7f4b5c0b4200 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.023+0000 7f4b43fff640 1 -- 172.21.15.37:0/596237702 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b4c018070 con 0x7f4b5c0b4200 2024-04-26T20:21:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.024+0000 7f4b43fff640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b4806c8f0 0x7f4b4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.024+0000 7f4b43fff640 1 -- 172.21.15.37:0/596237702 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4b4c08bb10 con 0x7f4b5c0b4200 2024-04-26T20:21:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.024+0000 7f4b5affd640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b4806c8f0 0x7f4b4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.024+0000 7f4b5affd640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b4806c8f0 0x7f4b4806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c1514f0 tx=0x7f4b50007cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.028+0000 7f4b43fff640 1 -- 172.21.15.37:0/596237702 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b5c1f75e0 con 0x7f4b5c0b4200 2024-04-26T20:21:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.253+0000 7f4b61c20640 1 -- 172.21.15.37:0/596237702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4b5c0cd670 con 0x7f4b5c0b4200 2024-04-26T20:21:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.895+0000 7f4b43fff640 1 -- 172.21.15.37:0/596237702 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f4b4c059f40 con 0x7f4b5c0b4200 2024-04-26T20:21:52.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 -- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b4806c8f0 msgr2=0x7f4b4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:52.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b4806c8f0 0x7f4b4806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c1514f0 tx=0x7f4b50007cc0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 -- 172.21.15.37:0/596237702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c0b4200 msgr2=0x7f4b5c150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:52.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c0b4200 0x7f4b5c150fb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c7f0 tx=0x7f4b4c00cbf0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 -- 172.21.15.37:0/596237702 shutdown_connections 2024-04-26T20:21:52.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4b4806c8f0 0x7f4b4806edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b5c15d320 0x7f4b5c1f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b5c0d1090 0x7f4b5c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 --2- 172.21.15.37:0/596237702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c0b4200 0x7f4b5c150fb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 -- 172.21.15.37:0/596237702 >> 172.21.15.37:0/596237702 conn(0x7f4b5c0d2f80 msgr2=0x7f4b5c071940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:52.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 -- 172.21.15.37:0/596237702 shutdown_connections 2024-04-26T20:21:52.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:52.898+0000 7f4b61c20640 1 -- 172.21.15.37:0/596237702 wait complete. 2024-04-26T20:21:52.912 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:21:52.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:21:52.962 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-26T20:21:52.961+0000 7f8519c9a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:21:53.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.018+0000 7f998c8c8640 1 Processor -- start 2024-04-26T20:21:53.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.018+0000 7f998c8c8640 1 -- start start 2024-04-26T20:21:53.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f998c8c8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99880d22e0 0x7f99880b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f998c8c8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f998815f760 0x7f99880b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f998c8c8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99880d3de0 con 0x7f998815f380 2024-04-26T20:21:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f998c8c8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f99880d43a0 con 0x7f998815f760 2024-04-26T20:21:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f998c8c8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f99880d40c0 con 0x7f99880d22e0 2024-04-26T20:21:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f9986d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f998815f760 0x7f99880b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f998659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f998815f380 0x7f99880d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52722/0 (socket says 172.21.15.37:52722) 2024-04-26T20:21:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f998659c640 1 -- 172.21.15.37:0/3839416501 learned_addr learned my addr 172.21.15.37:0/3839416501 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f997ffff640 1 --2- 172.21.15.37:0/3839416501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99880d22e0 0x7f99880b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f9986d9d640 1 -- 172.21.15.37:0/3839416501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99880d22e0 msgr2=0x7f99880b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f9986d9d640 1 --2- 172.21.15.37:0/3839416501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99880d22e0 0x7f99880b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f9986d9d640 1 -- 172.21.15.37:0/3839416501 >> v1:172.21.15.37:6789/0 conn(0x7f998815f380 legacy=0x7f99880d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f9986d9d640 1 -- 172.21.15.37:0/3839416501 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99880b8280 con 0x7f998815f760 2024-04-26T20:21:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.019+0000 7f9985d9b640 1 -- 172.21.15.37:0/3839416501 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4168401667 0 0) 0x7f99880d3de0 con 0x7f998815f380 2024-04-26T20:21:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.020+0000 7f9986d9d640 1 --2- 172.21.15.37:0/3839416501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998815f760 0x7f99880b8fc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9980009d90 tx=0x7f998002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9f7505defc51ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.020+0000 7f9985d9b640 1 -- 172.21.15.37:0/3839416501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980007740 con 0x7f998815f760 2024-04-26T20:21:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.020+0000 7f9985d9b640 1 -- 172.21.15.37:0/3839416501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99800078e0 con 0x7f998815f760 2024-04-26T20:21:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.020+0000 7f9985d9b640 1 -- 172.21.15.37:0/3839416501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980007cb0 con 0x7f998815f760 2024-04-26T20:21:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.021+0000 7f998c8c8640 1 -- 172.21.15.37:0/3839416501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998815f760 msgr2=0x7f99880b8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.021+0000 7f998c8c8640 1 --2- 172.21.15.37:0/3839416501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998815f760 0x7f99880b8fc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9980009d90 tx=0x7f998002a6f0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.021+0000 7f998c8c8640 1 -- 172.21.15.37:0/3839416501 shutdown_connections 2024-04-26T20:21:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.021+0000 7f998c8c8640 1 --2- 172.21.15.37:0/3839416501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998815f760 0x7f99880b8fc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.021+0000 7f998c8c8640 1 --2- 172.21.15.37:0/3839416501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99880d22e0 0x7f99880b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.021+0000 7f998c8c8640 1 -- 172.21.15.37:0/3839416501 >> 172.21.15.37:0/3839416501 conn(0x7f998809e560 msgr2=0x7f99880d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.021+0000 7f998c8c8640 1 -- 172.21.15.37:0/3839416501 shutdown_connections 2024-04-26T20:21:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.021+0000 7f998c8c8640 1 -- 172.21.15.37:0/3839416501 wait complete. 2024-04-26T20:21:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.022+0000 7f998c8c8640 1 Processor -- start 2024-04-26T20:21:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.022+0000 7f998c8c8640 1 -- start start 2024-04-26T20:21:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f998c8c8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99880d22e0 0x7f99881f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f998c8c8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998815f380 0x7f99881f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f998c8c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998815f760 0x7f99880ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f998c8c8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99880d1280 con 0x7f998815f760 2024-04-26T20:21:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f998c8c8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f99880d1100 con 0x7f99880d22e0 2024-04-26T20:21:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f998c8c8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f99880d5b50 con 0x7f998815f380 2024-04-26T20:21:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f9986d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998815f760 0x7f99880ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f9986d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998815f760 0x7f99880ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59496/0 (socket says 172.21.15.37:59496) 2024-04-26T20:21:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f9986d9d640 1 -- 172.21.15.37:0/2125568280 learned_addr learned my addr 172.21.15.37:0/2125568280 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f997ffff640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998815f380 0x7f99881f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f998659c640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99880d22e0 0x7f99881f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f9986d9d640 1 -- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998815f380 msgr2=0x7f99881f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f9986d9d640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998815f380 0x7f99881f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f9986d9d640 1 -- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99880d22e0 msgr2=0x7f99881f8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f9986d9d640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99880d22e0 0x7f99881f8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f9986d9d640 1 -- 172.21.15.37:0/2125568280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99880cb490 con 0x7f998815f760 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.023+0000 7f9986d9d640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998815f760 0x7f99880ca9f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9980009eb0 tx=0x7f9980004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.024+0000 7f997effd640 1 -- 172.21.15.37:0/2125568280 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998003f070 con 0x7f998815f760 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.024+0000 7f997effd640 1 -- 172.21.15.37:0/2125568280 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9980031c90 con 0x7f998815f760 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.024+0000 7f997effd640 1 -- 172.21.15.37:0/2125568280 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980044680 con 0x7f998815f760 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.024+0000 7f998c8c8640 1 -- 172.21.15.37:0/2125568280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99880cd730 con 0x7f998815f760 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.024+0000 7f998c8c8640 1 -- 172.21.15.37:0/2125568280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99880cdcf0 con 0x7f998815f760 2024-04-26T20:21:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.025+0000 7f998c8c8640 1 -- 172.21.15.37:0/2125568280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998806a360 con 0x7f998815f760 2024-04-26T20:21:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.025+0000 7f997effd640 1 -- 172.21.15.37:0/2125568280 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f998003b410 con 0x7f998815f760 2024-04-26T20:21:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.026+0000 7f997effd640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f996806c8f0 0x7f996806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.026+0000 7f997effd640 1 -- 172.21.15.37:0/2125568280 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f99800acd20 con 0x7f998815f760 2024-04-26T20:21:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.026+0000 7f998659c640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f996806c8f0 0x7f996806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.026+0000 7f998659c640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f996806c8f0 0x7f996806edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9978000f60 tx=0x7f9978007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.030+0000 7f997effd640 1 -- 172.21.15.37:0/2125568280 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f998004b070 con 0x7f998815f760 2024-04-26T20:21:53.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.252+0000 7f998c8c8640 1 -- 172.21.15.37:0/2125568280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99880c9dc0 con 0x7f998815f760 2024-04-26T20:21:53.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.253+0000 7f997effd640 1 -- 172.21.15.37:0/2125568280 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7f998007ab20 con 0x7f998815f760 2024-04-26T20:21:53.254 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:53.254 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2911741012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2911741012},{"type":"v1","addr":"172.21.15.134:6836","nonce":2911741012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1600483467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1600483467},{"type":"v1","addr":"172.21.15.37:6835","nonce":1600483467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2722614558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2722614558},{"type":"v1","addr":"172.21.15.134:6837","nonce":2722614558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:21:49.237855+0000","modified":"2024-04-26T20:21:52.255148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3920647125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3920647125},{"type":"v1","addr":"172.21.15.37:6834","nonce":3920647125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-04-26T20:21:53.254 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-26T20:21:53.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 -- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f996806c8f0 msgr2=0x7f996806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f996806c8f0 0x7f996806edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9978000f60 tx=0x7f9978007d50 comp rx=0 tx=0).stop 2024-04-26T20:21:53.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 -- 172.21.15.37:0/2125568280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998815f760 msgr2=0x7f99880ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998815f760 0x7f99880ca9f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9980009eb0 tx=0x7f9980004a40 comp rx=0 tx=0).stop 2024-04-26T20:21:53.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 -- 172.21.15.37:0/2125568280 shutdown_connections 2024-04-26T20:21:53.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998815f760 0x7f99880ca9f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f996806c8f0 0x7f996806edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998815f380 0x7f99881f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 --2- 172.21.15.37:0/2125568280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99880d22e0 0x7f99881f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 -- 172.21.15.37:0/2125568280 >> 172.21.15.37:0/2125568280 conn(0x7f998809e560 msgr2=0x7f998806ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:53.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 -- 172.21.15.37:0/2125568280 shutdown_connections 2024-04-26T20:21:53.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.255+0000 7f998c8c8640 1 -- 172.21.15.37:0/2125568280 wait complete. 2024-04-26T20:21:53.267 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:21:53.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.366+0000 7f947990a640 1 Processor -- start 2024-04-26T20:21:53.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.366+0000 7f947990a640 1 -- start start 2024-04-26T20:21:53.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.367+0000 7f947990a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f94740ce860 0x7f94740c7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.367+0000 7f947990a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94740ced60 con 0x7f94740d1c50 2024-04-26T20:21:53.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.367+0000 7f947990a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94740cf040 con 0x7f947415d310 2024-04-26T20:21:53.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.367+0000 7f947990a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f94740c6ab0 con 0x7f94740ce860 2024-04-26T20:21:53.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.367+0000 7f94737fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f94740ce860 0x7f94740c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.367+0000 7f94727fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f94740d1c50 0x7f94740c7200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52724/0 (socket says 172.21.15.37:52724) 2024-04-26T20:21:53.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.367+0000 7f94727fc640 1 -- 172.21.15.37:0/1846529242 learned_addr learned my addr 172.21.15.37:0/1846529242 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:53.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.367+0000 7f94737fe640 1 -- 172.21.15.37:0/1846529242 >> v1:172.21.15.134:6789/0 conn(0x7f947415d310 legacy=0x7f94740d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:53.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.367+0000 7f94737fe640 1 -- 172.21.15.37:0/1846529242 >> v1:172.21.15.37:6789/0 conn(0x7f94740d1c50 legacy=0x7f94740c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.368+0000 7f94737fe640 1 -- 172.21.15.37:0/1846529242 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94740cf320 con 0x7f94740ce860 2024-04-26T20:21:53.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.367+0000 7f9471ffb640 1 -- 172.21.15.37:0/1846529242 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3221456053 0 0) 0x7f94740ced60 con 0x7f94740d1c50 2024-04-26T20:21:53.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.368+0000 7f94737fe640 1 --2- 172.21.15.37:0/1846529242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94740ce860 0x7f94740c7910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f94740c6c30 tx=0x7f946402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a6e0e667584e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:53.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.368+0000 7f9471ffb640 1 -- 172.21.15.37:0/1846529242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464007740 con 0x7f94740ce860 2024-04-26T20:21:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.368+0000 7f9471ffb640 1 -- 172.21.15.37:0/1846529242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94640078e0 con 0x7f94740ce860 2024-04-26T20:21:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.368+0000 7f9471ffb640 1 -- 172.21.15.37:0/1846529242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464007cb0 con 0x7f94740ce860 2024-04-26T20:21:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.369+0000 7f947990a640 1 -- 172.21.15.37:0/1846529242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94740ce860 msgr2=0x7f94740c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.369+0000 7f947990a640 1 --2- 172.21.15.37:0/1846529242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94740ce860 0x7f94740c7910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f94740c6c30 tx=0x7f946402a6c0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.369+0000 7f947990a640 1 -- 172.21.15.37:0/1846529242 shutdown_connections 2024-04-26T20:21:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.369+0000 7f947990a640 1 --2- 172.21.15.37:0/1846529242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94740ce860 0x7f94740c7910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.369+0000 7f947990a640 1 -- 172.21.15.37:0/1846529242 >> 172.21.15.37:0/1846529242 conn(0x7f94740b4a50 msgr2=0x7f94740b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.369+0000 7f947990a640 1 -- 172.21.15.37:0/1846529242 shutdown_connections 2024-04-26T20:21:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.369+0000 7f947990a640 1 -- 172.21.15.37:0/1846529242 wait complete. 2024-04-26T20:21:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.370+0000 7f947990a640 1 Processor -- start 2024-04-26T20:21:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.370+0000 7f947990a640 1 -- start start 2024-04-26T20:21:53.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f947990a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94740ce860 0x7f9474150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f947990a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94740d1c50 0x7f9474153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f947990a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f947415d310 0x7f94740cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f947990a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f94740b5970 con 0x7f94740ce860 2024-04-26T20:21:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f947990a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94740b57f0 con 0x7f94740d1c50 2024-04-26T20:21:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f947990a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f947415d6f0 con 0x7f947415d310 2024-04-26T20:21:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f9472ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94740ce860 0x7f9474150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f9472ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94740ce860 0x7f9474150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59504/0 (socket says 172.21.15.37:59504) 2024-04-26T20:21:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f9472ffd640 1 -- 172.21.15.37:0/3386892068 learned_addr learned my addr 172.21.15.37:0/3386892068 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f94727fc640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94740d1c50 0x7f9474153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f94737fe640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f947415d310 0x7f94740cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f9472ffd640 1 -- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f947415d310 msgr2=0x7f94740cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f9472ffd640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f947415d310 0x7f94740cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f9472ffd640 1 -- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94740d1c50 msgr2=0x7f9474153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f9472ffd640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94740d1c50 0x7f9474153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.371+0000 7f9472ffd640 1 -- 172.21.15.37:0/3386892068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9474153840 con 0x7f94740ce860 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.372+0000 7f9472ffd640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94740ce860 0x7f9474150860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f946c00b3f0 tx=0x7f946c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.372+0000 7f946bfff640 1 -- 172.21.15.37:0/3386892068 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c015020 con 0x7f94740ce860 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.372+0000 7f946bfff640 1 -- 172.21.15.37:0/3386892068 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f946c011830 con 0x7f94740ce860 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.372+0000 7f947990a640 1 -- 172.21.15.37:0/3386892068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9474150e40 con 0x7f94740ce860 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.372+0000 7f946bfff640 1 -- 172.21.15.37:0/3386892068 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c011c20 con 0x7f94740ce860 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.372+0000 7f947990a640 1 -- 172.21.15.37:0/3386892068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94741516e0 con 0x7f94740ce860 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.373+0000 7f947990a640 1 -- 172.21.15.37:0/3386892068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94740693e0 con 0x7f94740ce860 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.373+0000 7f946bfff640 1 -- 172.21.15.37:0/3386892068 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f946c018020 con 0x7f94740ce860 2024-04-26T20:21:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.374+0000 7f946bfff640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f945406c8f0 0x7f945406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.374+0000 7f946bfff640 1 -- 172.21.15.37:0/3386892068 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f946c08b760 con 0x7f94740ce860 2024-04-26T20:21:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.374+0000 7f94727fc640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f945406c8f0 0x7f945406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.374+0000 7f94727fc640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f945406c8f0 0x7f945406edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f94740cd260 tx=0x7f945c007b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:53.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.378+0000 7f946bfff640 1 -- 172.21.15.37:0/3386892068 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f946c01f030 con 0x7f94740ce860 2024-04-26T20:21:53.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.565+0000 7f947990a640 1 -- 172.21.15.37:0/3386892068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94740c6400 con 0x7f94740ce860 2024-04-26T20:21:53.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.566+0000 7f946bfff640 1 -- 172.21.15.37:0/3386892068 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f9474153840 con 0x7f94740ce860 2024-04-26T20:21:53.568 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:53.568 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:21:53.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 -- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f945406c8f0 msgr2=0x7f945406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f945406c8f0 0x7f945406edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f94740cd260 tx=0x7f945c007b50 comp rx=0 tx=0).stop 2024-04-26T20:21:53.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 -- 172.21.15.37:0/3386892068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94740ce860 msgr2=0x7f9474150860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94740ce860 0x7f9474150860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f946c00b3f0 tx=0x7f946c00b8b0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 -- 172.21.15.37:0/3386892068 shutdown_connections 2024-04-26T20:21:53.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f945406c8f0 0x7f945406edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f947415d310 0x7f94740cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94740d1c50 0x7f9474153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 --2- 172.21.15.37:0/3386892068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94740ce860 0x7f9474150860 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 -- 172.21.15.37:0/3386892068 >> 172.21.15.37:0/3386892068 conn(0x7f94740b4a50 msgr2=0x7f94740c9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:53.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 -- 172.21.15.37:0/3386892068 shutdown_connections 2024-04-26T20:21:53.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.569+0000 7f947990a640 1 -- 172.21.15.37:0/3386892068 wait complete. 2024-04-26T20:21:53.581 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:21:53.581 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:21:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.680+0000 7f51ce052640 1 Processor -- start 2024-04-26T20:21:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.680+0000 7f51ce052640 1 -- start start 2024-04-26T20:21:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.681+0000 7f51ce052640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 0x7f51c80d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.681+0000 7f51ce052640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f51c815f760 0x7f51c80d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.681+0000 7f51ce052640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c80d3de0 con 0x7f51c80d5b50 2024-04-26T20:21:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.681+0000 7f51ce052640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f51c80d43a0 con 0x7f51c815f760 2024-04-26T20:21:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.681+0000 7f51ce052640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51c80d40c0 con 0x7f51c815f380 2024-04-26T20:21:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.681+0000 7f51c7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f51c80d5b50 0x7f51c80cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52728/0 (socket says 172.21.15.37:52728) 2024-04-26T20:21:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.681+0000 7f51c7fff640 1 -- 172.21.15.37:0/584017089 learned_addr learned my addr 172.21.15.37:0/584017089 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.681+0000 7f51c77fe640 1 --2- 172.21.15.37:0/584017089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 0x7f51c80d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.681+0000 7f51c6ffd640 1 --2- 172.21.15.37:0/584017089 >> v2:172.21.15.134:3300/0 conn(0x7f51c815f760 0x7f51c80d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.681+0000 7f51c67fc640 1 -- 172.21.15.37:0/584017089 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101296482 0 0) 0x7f51c80d3de0 con 0x7f51c80d5b50 2024-04-26T20:21:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.681+0000 7f51c67fc640 1 -- 172.21.15.37:0/584017089 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51b0003520 con 0x7f51c80d5b50 2024-04-26T20:21:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.682+0000 7f51c6ffd640 1 -- 172.21.15.37:0/584017089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 msgr2=0x7f51c80d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.682+0000 7f51c6ffd640 1 --2- 172.21.15.37:0/584017089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 0x7f51c80d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.682+0000 7f51c6ffd640 1 -- 172.21.15.37:0/584017089 >> v1:172.21.15.37:6789/0 conn(0x7f51c80d5b50 legacy=0x7f51c80cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.682+0000 7f51c67fc640 1 -- 172.21.15.37:0/584017089 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3207708273 0 0) 0x7f51b0003520 con 0x7f51c80d5b50 2024-04-26T20:21:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.682+0000 7f51c6ffd640 1 -- 172.21.15.37:0/584017089 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c80b8280 con 0x7f51c815f760 2024-04-26T20:21:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.682+0000 7f51c77fe640 1 --2- 172.21.15.37:0/584017089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 0x7f51c80d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:53.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.682+0000 7f51c6ffd640 1 --2- 172.21.15.37:0/584017089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c815f760 0x7f51c80d2110 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0097a0 tx=0x7f51bc02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d89f7d6b3b4bb244 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:53.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.683+0000 7f51c67fc640 1 -- 172.21.15.37:0/584017089 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc037070 con 0x7f51c815f760 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.683+0000 7f51c67fc640 1 -- 172.21.15.37:0/584017089 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51bc02ac30 con 0x7f51c815f760 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.683+0000 7f51c67fc640 1 -- 172.21.15.37:0/584017089 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c00019f0 con 0x7f51c815f760 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.683+0000 7f51ce052640 1 -- 172.21.15.37:0/584017089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c815f760 msgr2=0x7f51c80d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.683+0000 7f51ce052640 1 --2- 172.21.15.37:0/584017089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c815f760 0x7f51c80d2110 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0097a0 tx=0x7f51bc02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.683+0000 7f51ce052640 1 -- 172.21.15.37:0/584017089 shutdown_connections 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.683+0000 7f51ce052640 1 --2- 172.21.15.37:0/584017089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c815f760 0x7f51c80d2110 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.683+0000 7f51ce052640 1 --2- 172.21.15.37:0/584017089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 0x7f51c80d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.683+0000 7f51ce052640 1 -- 172.21.15.37:0/584017089 >> 172.21.15.37:0/584017089 conn(0x7f51c809e560 msgr2=0x7f51c80d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.683+0000 7f51ce052640 1 -- 172.21.15.37:0/584017089 shutdown_connections 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.683+0000 7f51ce052640 1 -- 172.21.15.37:0/584017089 wait complete. 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.684+0000 7f51ce052640 1 Processor -- start 2024-04-26T20:21:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.685+0000 7f51ce052640 1 -- start start 2024-04-26T20:21:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.685+0000 7f51ce052640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c80d5b50 0x7f51c81f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.685+0000 7f51ce052640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 0x7f51c8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.685+0000 7f51ce052640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51c815f760 0x7f51c81625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.685+0000 7f51ce052640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f51c80d1280 con 0x7f51c815f760 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.685+0000 7f51ce052640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51c80d1100 con 0x7f51c80d5b50 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.685+0000 7f51ce052640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51c80b9a90 con 0x7f51c815f380 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.685+0000 7f51c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51c815f760 0x7f51c81625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.685+0000 7f51c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51c815f760 0x7f51c81625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59518/0 (socket says 172.21.15.37:59518) 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.685+0000 7f51c7fff640 1 -- 172.21.15.37:0/2030757340 learned_addr learned my addr 172.21.15.37:0/2030757340 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.685+0000 7f51c77fe640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c80d5b50 0x7f51c81f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51c6ffd640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 0x7f51c8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51c7fff640 1 -- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 msgr2=0x7f51c8160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51c7fff640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 0x7f51c8160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51c7fff640 1 -- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c80d5b50 msgr2=0x7f51c81f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51c7fff640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c80d5b50 0x7f51c81f26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51c7fff640 1 -- 172.21.15.37:0/2030757340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c8161530 con 0x7f51c815f760 2024-04-26T20:21:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51c6ffd640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 0x7f51c8160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51c7fff640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51c815f760 0x7f51c81625d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f51c0001b60 tx=0x7f51c000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51affff640 1 -- 172.21.15.37:0/2030757340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0017070 con 0x7f51c815f760 2024-04-26T20:21:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51affff640 1 -- 172.21.15.37:0/2030757340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c0012400 con 0x7f51c815f760 2024-04-26T20:21:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51ce052640 1 -- 172.21.15.37:0/2030757340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c8161810 con 0x7f51c815f760 2024-04-26T20:21:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51affff640 1 -- 172.21.15.37:0/2030757340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c00127b0 con 0x7f51c815f760 2024-04-26T20:21:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.686+0000 7f51ce052640 1 -- 172.21.15.37:0/2030757340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c8161dd0 con 0x7f51c815f760 2024-04-26T20:21:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.687+0000 7f51ce052640 1 -- 172.21.15.37:0/2030757340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c806a360 con 0x7f51c815f760 2024-04-26T20:21:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.688+0000 7f51affff640 1 -- 172.21.15.37:0/2030757340 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51c001a070 con 0x7f51c815f760 2024-04-26T20:21:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.688+0000 7f51affff640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f51b406c8f0 0x7f51b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.688+0000 7f51affff640 1 -- 172.21.15.37:0/2030757340 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f51c008ba40 con 0x7f51c815f760 2024-04-26T20:21:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.688+0000 7f51c77fe640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f51b406c8f0 0x7f51b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.689+0000 7f51c77fe640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f51b406c8f0 0x7f51b406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009560 tx=0x7f51b8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.692+0000 7f51affff640 1 -- 172.21.15.37:0/2030757340 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51c00597a0 con 0x7f51c815f760 2024-04-26T20:21:53.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.876+0000 7f51ce052640 1 -- 172.21.15.37:0/2030757340 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51c806cfc0 con 0x7f51b406c8f0 2024-04-26T20:21:53.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.883+0000 7f51affff640 1 -- 172.21.15.37:0/2030757340 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f51c806cfc0 con 0x7f51b406c8f0 2024-04-26T20:21:53.887 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:53.888 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:21:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.885+0000 7f51ce052640 1 -- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f51b406c8f0 msgr2=0x7f51b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.885+0000 7f51ce052640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f51b406c8f0 0x7f51b406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009560 tx=0x7f51b8009190 comp rx=0 tx=0).stop 2024-04-26T20:21:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.885+0000 7f51ce052640 1 -- 172.21.15.37:0/2030757340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51c815f760 msgr2=0x7f51c81625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.885+0000 7f51ce052640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51c815f760 0x7f51c81625d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f51c0001b60 tx=0x7f51c000a430 comp rx=0 tx=0).stop 2024-04-26T20:21:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.885+0000 7f51ce052640 1 -- 172.21.15.37:0/2030757340 shutdown_connections 2024-04-26T20:21:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.885+0000 7f51ce052640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51c815f760 0x7f51c81625d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.885+0000 7f51ce052640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f51b406c8f0 0x7f51b406edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.885+0000 7f51ce052640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c815f380 0x7f51c8160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.885+0000 7f51ce052640 1 --2- 172.21.15.37:0/2030757340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c80d5b50 0x7f51c81f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.885+0000 7f51ce052640 1 -- 172.21.15.37:0/2030757340 >> 172.21.15.37:0/2030757340 conn(0x7f51c809e560 msgr2=0x7f51c806d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:53.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.885+0000 7f51ce052640 1 -- 172.21.15.37:0/2030757340 shutdown_connections 2024-04-26T20:21:53.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:53.886+0000 7f51ce052640 1 -- 172.21.15.37:0/2030757340 wait complete. 2024-04-26T20:21:53.897 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-26T20:21:53.335032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.478384"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317460+0000","last_change":"2024-04-26T20:21:49.899185+0000","last_active":"2024-04-26T20:21:51.317460+0000","last_peered":"2024-04-26T20:21:51.317460+0000","last_clean":"2024-04-26T20:21:51.317460+0000","last_became_active":"2024-04-26T20:21:49.898818+0000","last_became_peered":"2024-04-26T20:21:49.898818+0000","last_unstale":"2024-04-26T20:21:51.317460+0000","last_undegraded":"2024-04-26T20:21:51.317460+0000","last_fullsized":"2024-04-26T20:21:51.317460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:13.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.889053+0000","last_change":"2024-04-26T20:21:49.899484+0000","last_active":"2024-04-26T20:21:50.889053+0000","last_peered":"2024-04-26T20:21:50.889053+0000","last_clean":"2024-04-26T20:21:50.889053+0000","last_became_active":"2024-04-26T20:21:49.899091+0000","last_became_peered":"2024-04-26T20:21:49.899091+0000","last_unstale":"2024-04-26T20:21:50.889053+0000","last_undegraded":"2024-04-26T20:21:50.889053+0000","last_fullsized":"2024-04-26T20:21:50.889053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.887979+0000","last_change":"2024-04-26T20:21:49.881814+0000","last_active":"2024-04-26T20:21:50.887979+0000","last_peered":"2024-04-26T20:21:50.887979+0000","last_clean":"2024-04-26T20:21:50.887979+0000","last_became_active":"2024-04-26T20:21:49.881507+0000","last_became_peered":"2024-04-26T20:21:49.881507+0000","last_unstale":"2024-04-26T20:21:50.887979+0000","last_undegraded":"2024-04-26T20:21:50.887979+0000","last_fullsized":"2024-04-26T20:21:50.887979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:30.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318197+0000","last_change":"2024-04-26T20:21:49.896399+0000","last_active":"2024-04-26T20:21:51.318197+0000","last_peered":"2024-04-26T20:21:51.318197+0000","last_clean":"2024-04-26T20:21:51.318197+0000","last_became_active":"2024-04-26T20:21:49.896136+0000","last_became_peered":"2024-04-26T20:21:49.896136+0000","last_unstale":"2024-04-26T20:21:51.318197+0000","last_undegraded":"2024-04-26T20:21:51.318197+0000","last_fullsized":"2024-04-26T20:21:51.318197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.737636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317925+0000","last_change":"2024-04-26T20:21:49.898821+0000","last_active":"2024-04-26T20:21:51.317925+0000","last_peered":"2024-04-26T20:21:51.317925+0000","last_clean":"2024-04-26T20:21:51.317925+0000","last_became_active":"2024-04-26T20:21:49.898554+0000","last_became_peered":"2024-04-26T20:21:49.898554+0000","last_unstale":"2024-04-26T20:21:51.317925+0000","last_undegraded":"2024-04-26T20:21:51.317925+0000","last_fullsized":"2024-04-26T20:21:51.317925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:52.618628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316822+0000","last_change":"2024-04-26T20:21:49.878509+0000","last_active":"2024-04-26T20:21:51.316822+0000","last_peered":"2024-04-26T20:21:51.316822+0000","last_clean":"2024-04-26T20:21:51.316822+0000","last_became_active":"2024-04-26T20:21:49.878237+0000","last_became_peered":"2024-04-26T20:21:49.878237+0000","last_unstale":"2024-04-26T20:21:51.316822+0000","last_undegraded":"2024-04-26T20:21:51.316822+0000","last_fullsized":"2024-04-26T20:21:51.316822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316684+0000","last_change":"2024-04-26T20:21:49.898211+0000","last_active":"2024-04-26T20:21:51.316684+0000","last_peered":"2024-04-26T20:21:51.316684+0000","last_clean":"2024-04-26T20:21:51.316684+0000","last_became_active":"2024-04-26T20:21:49.897952+0000","last_became_peered":"2024-04-26T20:21:49.897952+0000","last_unstale":"2024-04-26T20:21:51.316684+0000","last_undegraded":"2024-04-26T20:21:51.316684+0000","last_fullsized":"2024-04-26T20:21:51.316684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316222+0000","last_change":"2024-04-26T20:21:49.892897+0000","last_active":"2024-04-26T20:21:51.316222+0000","last_peered":"2024-04-26T20:21:51.316222+0000","last_clean":"2024-04-26T20:21:51.316222+0000","last_became_active":"2024-04-26T20:21:49.890640+0000","last_became_peered":"2024-04-26T20:21:49.890640+0000","last_unstale":"2024-04-26T20:21:51.316222+0000","last_undegraded":"2024-04-26T20:21:51.316222+0000","last_fullsized":"2024-04-26T20:21:51.316222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:42.131574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317597+0000","last_change":"2024-04-26T20:21:49.901710+0000","last_active":"2024-04-26T20:21:51.317597+0000","last_peered":"2024-04-26T20:21:51.317597+0000","last_clean":"2024-04-26T20:21:51.317597+0000","last_became_active":"2024-04-26T20:21:49.901363+0000","last_became_peered":"2024-04-26T20:21:49.901363+0000","last_unstale":"2024-04-26T20:21:51.317597+0000","last_undegraded":"2024-04-26T20:21:51.317597+0000","last_fullsized":"2024-04-26T20:21:51.317597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:50.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891437+0000","last_change":"2024-04-26T20:21:49.900636+0000","last_active":"2024-04-26T20:21:50.891437+0000","last_peered":"2024-04-26T20:21:50.891437+0000","last_clean":"2024-04-26T20:21:50.891437+0000","last_became_active":"2024-04-26T20:21:49.900073+0000","last_became_peered":"2024-04-26T20:21:49.900073+0000","last_unstale":"2024-04-26T20:21:50.891437+0000","last_undegraded":"2024-04-26T20:21:50.891437+0000","last_fullsized":"2024-04-26T20:21:50.891437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891162+0000","last_change":"2024-04-26T20:21:49.892413+0000","last_active":"2024-04-26T20:21:50.891162+0000","last_peered":"2024-04-26T20:21:50.891162+0000","last_clean":"2024-04-26T20:21:50.891162+0000","last_became_active":"2024-04-26T20:21:49.892106+0000","last_became_peered":"2024-04-26T20:21:49.892106+0000","last_unstale":"2024-04-26T20:21:50.891162+0000","last_undegraded":"2024-04-26T20:21:50.891162+0000","last_fullsized":"2024-04-26T20:21:50.891162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317031+0000","last_change":"2024-04-26T20:21:49.894073+0000","last_active":"2024-04-26T20:21:51.317031+0000","last_peered":"2024-04-26T20:21:51.317031+0000","last_clean":"2024-04-26T20:21:51.317031+0000","last_became_active":"2024-04-26T20:21:49.893726+0000","last_became_peered":"2024-04-26T20:21:49.893726+0000","last_unstale":"2024-04-26T20:21:51.317031+0000","last_undegraded":"2024-04-26T20:21:51.317031+0000","last_fullsized":"2024-04-26T20:21:51.317031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:00.300212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316742+0000","last_change":"2024-04-26T20:21:49.895923+0000","last_active":"2024-04-26T20:21:51.316742+0000","last_peered":"2024-04-26T20:21:51.316742+0000","last_clean":"2024-04-26T20:21:51.316742+0000","last_became_active":"2024-04-26T20:21:49.895261+0000","last_became_peered":"2024-04-26T20:21:49.895261+0000","last_unstale":"2024-04-26T20:21:51.316742+0000","last_undegraded":"2024-04-26T20:21:51.316742+0000","last_fullsized":"2024-04-26T20:21:51.316742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.887942+0000","last_change":"2024-04-26T20:21:49.904360+0000","last_active":"2024-04-26T20:21:50.887942+0000","last_peered":"2024-04-26T20:21:50.887942+0000","last_clean":"2024-04-26T20:21:50.887942+0000","last_became_active":"2024-04-26T20:21:49.899936+0000","last_became_peered":"2024-04-26T20:21:49.899936+0000","last_unstale":"2024-04-26T20:21:50.887942+0000","last_undegraded":"2024-04-26T20:21:50.887942+0000","last_fullsized":"2024-04-26T20:21:50.887942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:55.987722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888860+0000","last_change":"2024-04-26T20:21:49.893445+0000","last_active":"2024-04-26T20:21:50.888860+0000","last_peered":"2024-04-26T20:21:50.888860+0000","last_clean":"2024-04-26T20:21:50.888860+0000","last_became_active":"2024-04-26T20:21:49.892704+0000","last_became_peered":"2024-04-26T20:21:49.892704+0000","last_unstale":"2024-04-26T20:21:50.888860+0000","last_undegraded":"2024-04-26T20:21:50.888860+0000","last_fullsized":"2024-04-26T20:21:50.888860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316915+0000","last_change":"2024-04-26T20:21:49.883172+0000","last_active":"2024-04-26T20:21:51.316915+0000","last_peered":"2024-04-26T20:21:51.316915+0000","last_clean":"2024-04-26T20:21:51.316915+0000","last_became_active":"2024-04-26T20:21:49.882916+0000","last_became_peered":"2024-04-26T20:21:49.882916+0000","last_unstale":"2024-04-26T20:21:51.316915+0000","last_undegraded":"2024-04-26T20:21:51.316915+0000","last_fullsized":"2024-04-26T20:21:51.316915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317389+0000","last_change":"2024-04-26T20:21:49.889742+0000","last_active":"2024-04-26T20:21:51.317389+0000","last_peered":"2024-04-26T20:21:51.317389+0000","last_clean":"2024-04-26T20:21:51.317389+0000","last_became_active":"2024-04-26T20:21:49.889417+0000","last_became_peered":"2024-04-26T20:21:49.889417+0000","last_unstale":"2024-04-26T20:21:51.317389+0000","last_undegraded":"2024-04-26T20:21:51.317389+0000","last_fullsized":"2024-04-26T20:21:51.317389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:53.734129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890942+0000","last_change":"2024-04-26T20:21:49.894387+0000","last_active":"2024-04-26T20:21:50.890942+0000","last_peered":"2024-04-26T20:21:50.890942+0000","last_clean":"2024-04-26T20:21:50.890942+0000","last_became_active":"2024-04-26T20:21:49.893464+0000","last_became_peered":"2024-04-26T20:21:49.893464+0000","last_unstale":"2024-04-26T20:21:50.890942+0000","last_undegraded":"2024-04-26T20:21:50.890942+0000","last_fullsized":"2024-04-26T20:21:50.890942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:26.689749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317646+0000","last_change":"2024-04-26T20:21:49.892977+0000","last_active":"2024-04-26T20:21:51.317646+0000","last_peered":"2024-04-26T20:21:51.317646+0000","last_clean":"2024-04-26T20:21:51.317646+0000","last_became_active":"2024-04-26T20:21:49.892709+0000","last_became_peered":"2024-04-26T20:21:49.892709+0000","last_unstale":"2024-04-26T20:21:51.317646+0000","last_undegraded":"2024-04-26T20:21:51.317646+0000","last_fullsized":"2024-04-26T20:21:51.317646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:44.689592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891273+0000","last_change":"2024-04-26T20:21:49.892717+0000","last_active":"2024-04-26T20:21:50.891273+0000","last_peered":"2024-04-26T20:21:50.891273+0000","last_clean":"2024-04-26T20:21:50.891273+0000","last_became_active":"2024-04-26T20:21:49.892029+0000","last_became_peered":"2024-04-26T20:21:49.892029+0000","last_unstale":"2024-04-26T20:21:50.891273+0000","last_undegraded":"2024-04-26T20:21:50.891273+0000","last_fullsized":"2024-04-26T20:21:50.891273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888152+0000","last_change":"2024-04-26T20:21:49.891937+0000","last_active":"2024-04-26T20:21:50.888152+0000","last_peered":"2024-04-26T20:21:50.888152+0000","last_clean":"2024-04-26T20:21:50.888152+0000","last_became_active":"2024-04-26T20:21:49.891631+0000","last_became_peered":"2024-04-26T20:21:49.891631+0000","last_unstale":"2024-04-26T20:21:50.888152+0000","last_undegraded":"2024-04-26T20:21:50.888152+0000","last_fullsized":"2024-04-26T20:21:50.888152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318190+0000","last_change":"2024-04-26T20:21:49.890285+0000","last_active":"2024-04-26T20:21:51.318190+0000","last_peered":"2024-04-26T20:21:51.318190+0000","last_clean":"2024-04-26T20:21:51.318190+0000","last_became_active":"2024-04-26T20:21:49.888800+0000","last_became_peered":"2024-04-26T20:21:49.888800+0000","last_unstale":"2024-04-26T20:21:51.318190+0000","last_undegraded":"2024-04-26T20:21:51.318190+0000","last_fullsized":"2024-04-26T20:21:51.318190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:32.476892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316465+0000","last_change":"2024-04-26T20:21:49.890345+0000","last_active":"2024-04-26T20:21:51.316465+0000","last_peered":"2024-04-26T20:21:51.316465+0000","last_clean":"2024-04-26T20:21:51.316465+0000","last_became_active":"2024-04-26T20:21:49.889999+0000","last_became_peered":"2024-04-26T20:21:49.889999+0000","last_unstale":"2024-04-26T20:21:51.316465+0000","last_undegraded":"2024-04-26T20:21:51.316465+0000","last_fullsized":"2024-04-26T20:21:51.316465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317064+0000","last_change":"2024-04-26T20:21:49.897497+0000","last_active":"2024-04-26T20:21:51.317064+0000","last_peered":"2024-04-26T20:21:51.317064+0000","last_clean":"2024-04-26T20:21:51.317064+0000","last_became_active":"2024-04-26T20:21:49.896968+0000","last_became_peered":"2024-04-26T20:21:49.896968+0000","last_unstale":"2024-04-26T20:21:51.317064+0000","last_undegraded":"2024-04-26T20:21:51.317064+0000","last_fullsized":"2024-04-26T20:21:51.317064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316953+0000","last_change":"2024-04-26T20:21:49.890916+0000","last_active":"2024-04-26T20:21:51.316953+0000","last_peered":"2024-04-26T20:21:51.316953+0000","last_clean":"2024-04-26T20:21:51.316953+0000","last_became_active":"2024-04-26T20:21:49.890517+0000","last_became_peered":"2024-04-26T20:21:49.890517+0000","last_unstale":"2024-04-26T20:21:51.316953+0000","last_undegraded":"2024-04-26T20:21:51.316953+0000","last_fullsized":"2024-04-26T20:21:51.316953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891447+0000","last_change":"2024-04-26T20:21:49.903237+0000","last_active":"2024-04-26T20:21:50.891447+0000","last_peered":"2024-04-26T20:21:50.891447+0000","last_clean":"2024-04-26T20:21:50.891447+0000","last_became_active":"2024-04-26T20:21:49.902924+0000","last_became_peered":"2024-04-26T20:21:49.902924+0000","last_unstale":"2024-04-26T20:21:50.891447+0000","last_undegraded":"2024-04-26T20:21:50.891447+0000","last_fullsized":"2024-04-26T20:21:50.891447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888809+0000","last_change":"2024-04-26T20:21:49.905100+0000","last_active":"2024-04-26T20:21:50.888809+0000","last_peered":"2024-04-26T20:21:50.888809+0000","last_clean":"2024-04-26T20:21:50.888809+0000","last_became_active":"2024-04-26T20:21:49.904037+0000","last_became_peered":"2024-04-26T20:21:49.904037+0000","last_unstale":"2024-04-26T20:21:50.888809+0000","last_undegraded":"2024-04-26T20:21:50.888809+0000","last_fullsized":"2024-04-26T20:21:50.888809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316677+0000","last_change":"2024-04-26T20:21:49.891939+0000","last_active":"2024-04-26T20:21:51.316677+0000","last_peered":"2024-04-26T20:21:51.316677+0000","last_clean":"2024-04-26T20:21:51.316677+0000","last_became_active":"2024-04-26T20:21:49.891662+0000","last_became_peered":"2024-04-26T20:21:49.891662+0000","last_unstale":"2024-04-26T20:21:51.316677+0000","last_undegraded":"2024-04-26T20:21:51.316677+0000","last_fullsized":"2024-04-26T20:21:51.316677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888299+0000","last_change":"2024-04-26T20:21:49.895730+0000","last_active":"2024-04-26T20:21:50.888299+0000","last_peered":"2024-04-26T20:21:50.888299+0000","last_clean":"2024-04-26T20:21:50.888299+0000","last_became_active":"2024-04-26T20:21:49.895244+0000","last_became_peered":"2024-04-26T20:21:49.895244+0000","last_unstale":"2024-04-26T20:21:50.888299+0000","last_undegraded":"2024-04-26T20:21:50.888299+0000","last_fullsized":"2024-04-26T20:21:50.888299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318438+0000","last_change":"2024-04-26T20:21:49.896367+0000","last_active":"2024-04-26T20:21:51.318438+0000","last_peered":"2024-04-26T20:21:51.318438+0000","last_clean":"2024-04-26T20:21:51.318438+0000","last_became_active":"2024-04-26T20:21:49.895963+0000","last_became_peered":"2024-04-26T20:21:49.895963+0000","last_unstale":"2024-04-26T20:21:51.318438+0000","last_undegraded":"2024-04-26T20:21:51.318438+0000","last_fullsized":"2024-04-26T20:21:51.318438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:57.212796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890871+0000","last_change":"2024-04-26T20:21:49.880643+0000","last_active":"2024-04-26T20:21:50.890871+0000","last_peered":"2024-04-26T20:21:50.890871+0000","last_clean":"2024-04-26T20:21:50.890871+0000","last_became_active":"2024-04-26T20:21:49.880295+0000","last_became_peered":"2024-04-26T20:21:49.880295+0000","last_unstale":"2024-04-26T20:21:50.890871+0000","last_undegraded":"2024-04-26T20:21:50.890871+0000","last_fullsized":"2024-04-26T20:21:50.890871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:40.680121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891215+0000","last_change":"2024-04-26T20:21:49.906308+0000","last_active":"2024-04-26T20:21:50.891215+0000","last_peered":"2024-04-26T20:21:50.891215+0000","last_clean":"2024-04-26T20:21:50.891215+0000","last_became_active":"2024-04-26T20:21:49.906092+0000","last_became_peered":"2024-04-26T20:21:49.906092+0000","last_unstale":"2024-04-26T20:21:50.891215+0000","last_undegraded":"2024-04-26T20:21:50.891215+0000","last_fullsized":"2024-04-26T20:21:50.891215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318011+0000","last_change":"2024-04-26T20:21:49.901086+0000","last_active":"2024-04-26T20:21:51.318011+0000","last_peered":"2024-04-26T20:21:51.318011+0000","last_clean":"2024-04-26T20:21:51.318011+0000","last_became_active":"2024-04-26T20:21:49.900797+0000","last_became_peered":"2024-04-26T20:21:49.900797+0000","last_unstale":"2024-04-26T20:21:51.318011+0000","last_undegraded":"2024-04-26T20:21:51.318011+0000","last_fullsized":"2024-04-26T20:21:51.318011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:26.689749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316705+0000","last_change":"2024-04-26T20:21:48.880856+0000","last_active":"2024-04-26T20:21:51.316705+0000","last_peered":"2024-04-26T20:21:51.316705+0000","last_clean":"2024-04-26T20:21:51.316705+0000","last_became_active":"2024-04-26T20:21:48.880512+0000","last_became_peered":"2024-04-26T20:21:48.880512+0000","last_unstale":"2024-04-26T20:21:51.316705+0000","last_undegraded":"2024-04-26T20:21:51.316705+0000","last_fullsized":"2024-04-26T20:21:51.316705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888381+0000","last_change":"2024-04-26T20:21:49.889888+0000","last_active":"2024-04-26T20:21:50.888381+0000","last_peered":"2024-04-26T20:21:50.888381+0000","last_clean":"2024-04-26T20:21:50.888381+0000","last_became_active":"2024-04-26T20:21:49.889451+0000","last_became_peered":"2024-04-26T20:21:49.889451+0000","last_unstale":"2024-04-26T20:21:50.888381+0000","last_undegraded":"2024-04-26T20:21:50.888381+0000","last_fullsized":"2024-04-26T20:21:50.888381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891210+0000","last_change":"2024-04-26T20:21:48.880648+0000","last_active":"2024-04-26T20:21:50.891210+0000","last_peered":"2024-04-26T20:21:50.891210+0000","last_clean":"2024-04-26T20:21:50.891210+0000","last_became_active":"2024-04-26T20:21:48.880389+0000","last_became_peered":"2024-04-26T20:21:48.880389+0000","last_unstale":"2024-04-26T20:21:50.891210+0000","last_undegraded":"2024-04-26T20:21:50.891210+0000","last_fullsized":"2024-04-26T20:21:50.891210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316415+0000","last_change":"2024-04-26T20:21:48.873199+0000","last_active":"2024-04-26T20:21:51.316415+0000","last_peered":"2024-04-26T20:21:51.316415+0000","last_clean":"2024-04-26T20:21:51.316415+0000","last_became_active":"2024-04-26T20:21:48.872868+0000","last_became_peered":"2024-04-26T20:21:48.872868+0000","last_unstale":"2024-04-26T20:21:51.316415+0000","last_undegraded":"2024-04-26T20:21:51.316415+0000","last_fullsized":"2024-04-26T20:21:51.316415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316315+0000","last_change":"2024-04-26T20:21:49.897450+0000","last_active":"2024-04-26T20:21:51.316315+0000","last_peered":"2024-04-26T20:21:51.316315+0000","last_clean":"2024-04-26T20:21:51.316315+0000","last_became_active":"2024-04-26T20:21:49.897165+0000","last_became_peered":"2024-04-26T20:21:49.897165+0000","last_unstale":"2024-04-26T20:21:51.316315+0000","last_undegraded":"2024-04-26T20:21:51.316315+0000","last_fullsized":"2024-04-26T20:21:51.316315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317170+0000","last_change":"2024-04-26T20:21:48.881023+0000","last_active":"2024-04-26T20:21:51.317170+0000","last_peered":"2024-04-26T20:21:51.317170+0000","last_clean":"2024-04-26T20:21:51.317170+0000","last_became_active":"2024-04-26T20:21:48.880610+0000","last_became_peered":"2024-04-26T20:21:48.880610+0000","last_unstale":"2024-04-26T20:21:51.317170+0000","last_undegraded":"2024-04-26T20:21:51.317170+0000","last_fullsized":"2024-04-26T20:21:51.317170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888715+0000","last_change":"2024-04-26T20:21:49.905734+0000","last_active":"2024-04-26T20:21:50.888715+0000","last_peered":"2024-04-26T20:21:50.888715+0000","last_clean":"2024-04-26T20:21:50.888715+0000","last_became_active":"2024-04-26T20:21:49.905424+0000","last_became_peered":"2024-04-26T20:21:49.905424+0000","last_unstale":"2024-04-26T20:21:50.888715+0000","last_undegraded":"2024-04-26T20:21:50.888715+0000","last_fullsized":"2024-04-26T20:21:50.888715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891379+0000","last_change":"2024-04-26T20:21:48.877043+0000","last_active":"2024-04-26T20:21:50.891379+0000","last_peered":"2024-04-26T20:21:50.891379+0000","last_clean":"2024-04-26T20:21:50.891379+0000","last_became_active":"2024-04-26T20:21:48.876721+0000","last_became_peered":"2024-04-26T20:21:48.876721+0000","last_unstale":"2024-04-26T20:21:50.891379+0000","last_undegraded":"2024-04-26T20:21:50.891379+0000","last_fullsized":"2024-04-26T20:21:50.891379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317752+0000","last_change":"2024-04-26T20:21:49.890323+0000","last_active":"2024-04-26T20:21:51.317752+0000","last_peered":"2024-04-26T20:21:51.317752+0000","last_clean":"2024-04-26T20:21:51.317752+0000","last_became_active":"2024-04-26T20:21:49.889790+0000","last_became_peered":"2024-04-26T20:21:49.889790+0000","last_unstale":"2024-04-26T20:21:51.317752+0000","last_undegraded":"2024-04-26T20:21:51.317752+0000","last_fullsized":"2024-04-26T20:21:51.317752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:27.648550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317863+0000","last_change":"2024-04-26T20:21:48.872759+0000","last_active":"2024-04-26T20:21:51.317863+0000","last_peered":"2024-04-26T20:21:51.317863+0000","last_clean":"2024-04-26T20:21:51.317863+0000","last_became_active":"2024-04-26T20:21:48.872461+0000","last_became_peered":"2024-04-26T20:21:48.872461+0000","last_unstale":"2024-04-26T20:21:51.317863+0000","last_undegraded":"2024-04-26T20:21:51.317863+0000","last_fullsized":"2024-04-26T20:21:51.317863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888264+0000","last_change":"2024-04-26T20:21:49.885391+0000","last_active":"2024-04-26T20:21:50.888264+0000","last_peered":"2024-04-26T20:21:50.888264+0000","last_clean":"2024-04-26T20:21:50.888264+0000","last_became_active":"2024-04-26T20:21:49.884866+0000","last_became_peered":"2024-04-26T20:21:49.884866+0000","last_unstale":"2024-04-26T20:21:50.888264+0000","last_undegraded":"2024-04-26T20:21:50.888264+0000","last_fullsized":"2024-04-26T20:21:50.888264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888983+0000","last_change":"2024-04-26T20:21:48.873334+0000","last_active":"2024-04-26T20:21:50.888983+0000","last_peered":"2024-04-26T20:21:50.888983+0000","last_clean":"2024-04-26T20:21:50.888983+0000","last_became_active":"2024-04-26T20:21:48.872910+0000","last_became_peered":"2024-04-26T20:21:48.872910+0000","last_unstale":"2024-04-26T20:21:50.888983+0000","last_undegraded":"2024-04-26T20:21:50.888983+0000","last_fullsized":"2024-04-26T20:21:50.888983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:39.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317180+0000","last_change":"2024-04-26T20:21:49.889288+0000","last_active":"2024-04-26T20:21:51.317180+0000","last_peered":"2024-04-26T20:21:51.317180+0000","last_clean":"2024-04-26T20:21:51.317180+0000","last_became_active":"2024-04-26T20:21:49.889007+0000","last_became_peered":"2024-04-26T20:21:49.889007+0000","last_unstale":"2024-04-26T20:21:51.317180+0000","last_undegraded":"2024-04-26T20:21:51.317180+0000","last_fullsized":"2024-04-26T20:21:51.317180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:02.548978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891213+0000","last_change":"2024-04-26T20:21:48.875893+0000","last_active":"2024-04-26T20:21:50.891213+0000","last_peered":"2024-04-26T20:21:50.891213+0000","last_clean":"2024-04-26T20:21:50.891213+0000","last_became_active":"2024-04-26T20:21:48.875620+0000","last_became_peered":"2024-04-26T20:21:48.875620+0000","last_unstale":"2024-04-26T20:21:50.891213+0000","last_undegraded":"2024-04-26T20:21:50.891213+0000","last_fullsized":"2024-04-26T20:21:50.891213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317151+0000","last_change":"2024-04-26T20:21:49.895277+0000","last_active":"2024-04-26T20:21:51.317151+0000","last_peered":"2024-04-26T20:21:51.317151+0000","last_clean":"2024-04-26T20:21:51.317151+0000","last_became_active":"2024-04-26T20:21:49.894988+0000","last_became_peered":"2024-04-26T20:21:49.894988+0000","last_unstale":"2024-04-26T20:21:51.317151+0000","last_undegraded":"2024-04-26T20:21:51.317151+0000","last_fullsized":"2024-04-26T20:21:51.317151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:32.476892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888386+0000","last_change":"2024-04-26T20:21:48.881818+0000","last_active":"2024-04-26T20:21:50.888386+0000","last_peered":"2024-04-26T20:21:50.888386+0000","last_clean":"2024-04-26T20:21:50.888386+0000","last_became_active":"2024-04-26T20:21:48.881426+0000","last_became_peered":"2024-04-26T20:21:48.881426+0000","last_unstale":"2024-04-26T20:21:50.888386+0000","last_undegraded":"2024-04-26T20:21:50.888386+0000","last_fullsized":"2024-04-26T20:21:50.888386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:10.468097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888081+0000","last_change":"2024-04-26T20:21:49.891143+0000","last_active":"2024-04-26T20:21:50.888081+0000","last_peered":"2024-04-26T20:21:50.888081+0000","last_clean":"2024-04-26T20:21:50.888081+0000","last_became_active":"2024-04-26T20:21:49.890710+0000","last_became_peered":"2024-04-26T20:21:49.890710+0000","last_unstale":"2024-04-26T20:21:50.888081+0000","last_undegraded":"2024-04-26T20:21:50.888081+0000","last_fullsized":"2024-04-26T20:21:50.888081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318330+0000","last_change":"2024-04-26T20:21:48.872822+0000","last_active":"2024-04-26T20:21:51.318330+0000","last_peered":"2024-04-26T20:21:51.318330+0000","last_clean":"2024-04-26T20:21:51.318330+0000","last_became_active":"2024-04-26T20:21:48.872542+0000","last_became_peered":"2024-04-26T20:21:48.872542+0000","last_unstale":"2024-04-26T20:21:51.318330+0000","last_undegraded":"2024-04-26T20:21:51.318330+0000","last_fullsized":"2024-04-26T20:21:51.318330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318093+0000","last_change":"2024-04-26T20:21:49.894321+0000","last_active":"2024-04-26T20:21:51.318093+0000","last_peered":"2024-04-26T20:21:51.318093+0000","last_clean":"2024-04-26T20:21:51.318093+0000","last_became_active":"2024-04-26T20:21:49.893996+0000","last_became_peered":"2024-04-26T20:21:49.893996+0000","last_unstale":"2024-04-26T20:21:51.318093+0000","last_undegraded":"2024-04-26T20:21:51.318093+0000","last_fullsized":"2024-04-26T20:21:51.318093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:52.770156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891066+0000","last_change":"2024-04-26T20:21:48.879859+0000","last_active":"2024-04-26T20:21:50.891066+0000","last_peered":"2024-04-26T20:21:50.891066+0000","last_clean":"2024-04-26T20:21:50.891066+0000","last_became_active":"2024-04-26T20:21:48.879557+0000","last_became_peered":"2024-04-26T20:21:48.879557+0000","last_unstale":"2024-04-26T20:21:50.891066+0000","last_undegraded":"2024-04-26T20:21:50.891066+0000","last_fullsized":"2024-04-26T20:21:50.891066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891233+0000","last_change":"2024-04-26T20:21:49.902684+0000","last_active":"2024-04-26T20:21:50.891233+0000","last_peered":"2024-04-26T20:21:50.891233+0000","last_clean":"2024-04-26T20:21:50.891233+0000","last_became_active":"2024-04-26T20:21:49.902380+0000","last_became_peered":"2024-04-26T20:21:49.902380+0000","last_unstale":"2024-04-26T20:21:50.891233+0000","last_undegraded":"2024-04-26T20:21:50.891233+0000","last_fullsized":"2024-04-26T20:21:50.891233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890777+0000","last_change":"2024-04-26T20:21:48.877680+0000","last_active":"2024-04-26T20:21:50.890777+0000","last_peered":"2024-04-26T20:21:50.890777+0000","last_clean":"2024-04-26T20:21:50.890777+0000","last_became_active":"2024-04-26T20:21:48.877390+0000","last_became_peered":"2024-04-26T20:21:48.877390+0000","last_unstale":"2024-04-26T20:21:50.890777+0000","last_undegraded":"2024-04-26T20:21:50.890777+0000","last_fullsized":"2024-04-26T20:21:50.890777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316851+0000","last_change":"2024-04-26T20:21:49.896604+0000","last_active":"2024-04-26T20:21:51.316851+0000","last_peered":"2024-04-26T20:21:51.316851+0000","last_clean":"2024-04-26T20:21:51.316851+0000","last_became_active":"2024-04-26T20:21:49.894327+0000","last_became_peered":"2024-04-26T20:21:49.894327+0000","last_unstale":"2024-04-26T20:21:51.316851+0000","last_undegraded":"2024-04-26T20:21:51.316851+0000","last_fullsized":"2024-04-26T20:21:51.316851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:46.887612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318312+0000","last_change":"2024-04-26T20:21:48.880981+0000","last_active":"2024-04-26T20:21:51.318312+0000","last_peered":"2024-04-26T20:21:51.318312+0000","last_clean":"2024-04-26T20:21:51.318312+0000","last_became_active":"2024-04-26T20:21:48.880662+0000","last_became_peered":"2024-04-26T20:21:48.880662+0000","last_unstale":"2024-04-26T20:21:51.318312+0000","last_undegraded":"2024-04-26T20:21:51.318312+0000","last_fullsized":"2024-04-26T20:21:51.318312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:12.587571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317595+0000","last_change":"2024-04-26T20:21:49.894131+0000","last_active":"2024-04-26T20:21:51.317595+0000","last_peered":"2024-04-26T20:21:51.317595+0000","last_clean":"2024-04-26T20:21:51.317595+0000","last_became_active":"2024-04-26T20:21:49.893827+0000","last_became_peered":"2024-04-26T20:21:49.893827+0000","last_unstale":"2024-04-26T20:21:51.317595+0000","last_undegraded":"2024-04-26T20:21:51.317595+0000","last_fullsized":"2024-04-26T20:21:51.317595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:57.212796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890988+0000","last_change":"2024-04-26T20:21:48.875201+0000","last_active":"2024-04-26T20:21:50.890988+0000","last_peered":"2024-04-26T20:21:50.890988+0000","last_clean":"2024-04-26T20:21:50.890988+0000","last_became_active":"2024-04-26T20:21:48.874859+0000","last_became_peered":"2024-04-26T20:21:48.874859+0000","last_unstale":"2024-04-26T20:21:50.890988+0000","last_undegraded":"2024-04-26T20:21:50.890988+0000","last_fullsized":"2024-04-26T20:21:50.890988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317768+0000","last_change":"2024-04-26T20:21:49.893283+0000","last_active":"2024-04-26T20:21:51.317768+0000","last_peered":"2024-04-26T20:21:51.317768+0000","last_clean":"2024-04-26T20:21:51.317768+0000","last_became_active":"2024-04-26T20:21:49.893006+0000","last_became_peered":"2024-04-26T20:21:49.893006+0000","last_unstale":"2024-04-26T20:21:51.317768+0000","last_undegraded":"2024-04-26T20:21:51.317768+0000","last_fullsized":"2024-04-26T20:21:51.317768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:57.391564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888957+0000","last_change":"2024-04-26T20:21:48.880076+0000","last_active":"2024-04-26T20:21:50.888957+0000","last_peered":"2024-04-26T20:21:50.888957+0000","last_clean":"2024-04-26T20:21:50.888957+0000","last_became_active":"2024-04-26T20:21:48.879805+0000","last_became_peered":"2024-04-26T20:21:48.879805+0000","last_unstale":"2024-04-26T20:21:50.888957+0000","last_undegraded":"2024-04-26T20:21:50.888957+0000","last_fullsized":"2024-04-26T20:21:50.888957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:25.685618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318491+0000","last_change":"2024-04-26T20:21:49.895219+0000","last_active":"2024-04-26T20:21:51.318491+0000","last_peered":"2024-04-26T20:21:51.318491+0000","last_clean":"2024-04-26T20:21:51.318491+0000","last_became_active":"2024-04-26T20:21:49.894965+0000","last_became_peered":"2024-04-26T20:21:49.894965+0000","last_unstale":"2024-04-26T20:21:51.318491+0000","last_undegraded":"2024-04-26T20:21:51.318491+0000","last_fullsized":"2024-04-26T20:21:51.318491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:50.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890958+0000","last_change":"2024-04-26T20:21:48.878737+0000","last_active":"2024-04-26T20:21:50.890958+0000","last_peered":"2024-04-26T20:21:50.890958+0000","last_clean":"2024-04-26T20:21:50.890958+0000","last_became_active":"2024-04-26T20:21:48.878184+0000","last_became_peered":"2024-04-26T20:21:48.878184+0000","last_unstale":"2024-04-26T20:21:50.890958+0000","last_undegraded":"2024-04-26T20:21:50.890958+0000","last_fullsized":"2024-04-26T20:21:50.890958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:40:48.030416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317314+0000","last_change":"2024-04-26T20:21:49.897731+0000","last_active":"2024-04-26T20:21:51.317314+0000","last_peered":"2024-04-26T20:21:51.317314+0000","last_clean":"2024-04-26T20:21:51.317314+0000","last_became_active":"2024-04-26T20:21:49.897398+0000","last_became_peered":"2024-04-26T20:21:49.897398+0000","last_unstale":"2024-04-26T20:21:51.317314+0000","last_undegraded":"2024-04-26T20:21:51.317314+0000","last_fullsized":"2024-04-26T20:21:51.317314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.737636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317884+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:21:51.317884+0000","last_peered":"2024-04-26T20:21:51.317884+0000","last_clean":"2024-04-26T20:21:51.317884+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:21:51.317884+0000","last_undegraded":"2024-04-26T20:21:51.317884+0000","last_fullsized":"2024-04-26T20:21:51.317884+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317283+0000","last_change":"2024-04-26T20:21:48.873546+0000","last_active":"2024-04-26T20:21:51.317283+0000","last_peered":"2024-04-26T20:21:51.317283+0000","last_clean":"2024-04-26T20:21:51.317283+0000","last_became_active":"2024-04-26T20:21:48.873258+0000","last_became_peered":"2024-04-26T20:21:48.873258+0000","last_unstale":"2024-04-26T20:21:51.317283+0000","last_undegraded":"2024-04-26T20:21:51.317283+0000","last_fullsized":"2024-04-26T20:21:51.317283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316468+0000","last_change":"2024-04-26T20:21:49.892620+0000","last_active":"2024-04-26T20:21:51.316468+0000","last_peered":"2024-04-26T20:21:51.316468+0000","last_clean":"2024-04-26T20:21:51.316468+0000","last_became_active":"2024-04-26T20:21:49.891979+0000","last_became_peered":"2024-04-26T20:21:49.891979+0000","last_unstale":"2024-04-26T20:21:51.316468+0000","last_undegraded":"2024-04-26T20:21:51.316468+0000","last_fullsized":"2024-04-26T20:21:51.316468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317818+0000","last_change":"2024-04-26T20:21:48.883411+0000","last_active":"2024-04-26T20:21:51.317818+0000","last_peered":"2024-04-26T20:21:51.317818+0000","last_clean":"2024-04-26T20:21:51.317818+0000","last_became_active":"2024-04-26T20:21:48.883156+0000","last_became_peered":"2024-04-26T20:21:48.883156+0000","last_unstale":"2024-04-26T20:21:51.317818+0000","last_undegraded":"2024-04-26T20:21:51.317818+0000","last_fullsized":"2024-04-26T20:21:51.317818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:39.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888021+0000","last_change":"2024-04-26T20:21:49.889027+0000","last_active":"2024-04-26T20:21:50.888021+0000","last_peered":"2024-04-26T20:21:50.888021+0000","last_clean":"2024-04-26T20:21:50.888021+0000","last_became_active":"2024-04-26T20:21:49.888730+0000","last_became_peered":"2024-04-26T20:21:49.888730+0000","last_unstale":"2024-04-26T20:21:50.888021+0000","last_undegraded":"2024-04-26T20:21:50.888021+0000","last_fullsized":"2024-04-26T20:21:50.888021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890869+0000","last_change":"2024-04-26T20:21:48.875168+0000","last_active":"2024-04-26T20:21:50.890869+0000","last_peered":"2024-04-26T20:21:50.890869+0000","last_clean":"2024-04-26T20:21:50.890869+0000","last_became_active":"2024-04-26T20:21:48.874839+0000","last_became_peered":"2024-04-26T20:21:48.874839+0000","last_unstale":"2024-04-26T20:21:50.890869+0000","last_undegraded":"2024-04-26T20:21:50.890869+0000","last_fullsized":"2024-04-26T20:21:50.890869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888697+0000","last_change":"2024-04-26T20:21:49.895165+0000","last_active":"2024-04-26T20:21:50.888697+0000","last_peered":"2024-04-26T20:21:50.888697+0000","last_clean":"2024-04-26T20:21:50.888697+0000","last_became_active":"2024-04-26T20:21:49.894872+0000","last_became_peered":"2024-04-26T20:21:49.894872+0000","last_unstale":"2024-04-26T20:21:50.888697+0000","last_undegraded":"2024-04-26T20:21:50.888697+0000","last_fullsized":"2024-04-26T20:21:50.888697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318427+0000","last_change":"2024-04-26T20:21:48.874437+0000","last_active":"2024-04-26T20:21:51.318427+0000","last_peered":"2024-04-26T20:21:51.318427+0000","last_clean":"2024-04-26T20:21:51.318427+0000","last_became_active":"2024-04-26T20:21:48.874165+0000","last_became_peered":"2024-04-26T20:21:48.874165+0000","last_unstale":"2024-04-26T20:21:51.318427+0000","last_undegraded":"2024-04-26T20:21:51.318427+0000","last_fullsized":"2024-04-26T20:21:51.318427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317263+0000","last_change":"2024-04-26T20:21:49.894448+0000","last_active":"2024-04-26T20:21:51.317263+0000","last_peered":"2024-04-26T20:21:51.317263+0000","last_clean":"2024-04-26T20:21:51.317263+0000","last_became_active":"2024-04-26T20:21:49.894119+0000","last_became_peered":"2024-04-26T20:21:49.894119+0000","last_unstale":"2024-04-26T20:21:51.317263+0000","last_undegraded":"2024-04-26T20:21:51.317263+0000","last_fullsized":"2024-04-26T20:21:51.317263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:40.680121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888896+0000","last_change":"2024-04-26T20:21:48.874713+0000","last_active":"2024-04-26T20:21:50.888896+0000","last_peered":"2024-04-26T20:21:50.888896+0000","last_clean":"2024-04-26T20:21:50.888896+0000","last_became_active":"2024-04-26T20:21:48.874414+0000","last_became_peered":"2024-04-26T20:21:48.874414+0000","last_unstale":"2024-04-26T20:21:50.888896+0000","last_undegraded":"2024-04-26T20:21:50.888896+0000","last_fullsized":"2024-04-26T20:21:50.888896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317980+0000","last_change":"2024-04-26T20:21:49.895171+0000","last_active":"2024-04-26T20:21:51.317980+0000","last_peered":"2024-04-26T20:21:51.317980+0000","last_clean":"2024-04-26T20:21:51.317980+0000","last_became_active":"2024-04-26T20:21:49.894889+0000","last_became_peered":"2024-04-26T20:21:49.894889+0000","last_unstale":"2024-04-26T20:21:51.317980+0000","last_undegraded":"2024-04-26T20:21:51.317980+0000","last_fullsized":"2024-04-26T20:21:51.317980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:46.887612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317633+0000","last_change":"2024-04-26T20:21:48.882622+0000","last_active":"2024-04-26T20:21:51.317633+0000","last_peered":"2024-04-26T20:21:51.317633+0000","last_clean":"2024-04-26T20:21:51.317633+0000","last_became_active":"2024-04-26T20:21:48.882344+0000","last_became_peered":"2024-04-26T20:21:48.882344+0000","last_unstale":"2024-04-26T20:21:51.317633+0000","last_undegraded":"2024-04-26T20:21:51.317633+0000","last_fullsized":"2024-04-26T20:21:51.317633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:35.034364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888822+0000","last_change":"2024-04-26T20:21:49.894444+0000","last_active":"2024-04-26T20:21:50.888822+0000","last_peered":"2024-04-26T20:21:50.888822+0000","last_clean":"2024-04-26T20:21:50.888822+0000","last_became_active":"2024-04-26T20:21:49.894111+0000","last_became_peered":"2024-04-26T20:21:49.894111+0000","last_unstale":"2024-04-26T20:21:50.888822+0000","last_undegraded":"2024-04-26T20:21:50.888822+0000","last_fullsized":"2024-04-26T20:21:50.888822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890978+0000","last_change":"2024-04-26T20:21:48.872831+0000","last_active":"2024-04-26T20:21:50.890978+0000","last_peered":"2024-04-26T20:21:50.890978+0000","last_clean":"2024-04-26T20:21:50.890978+0000","last_became_active":"2024-04-26T20:21:48.872523+0000","last_became_peered":"2024-04-26T20:21:48.872523+0000","last_unstale":"2024-04-26T20:21:50.890978+0000","last_undegraded":"2024-04-26T20:21:50.890978+0000","last_fullsized":"2024-04-26T20:21:50.890978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316932+0000","last_change":"2024-04-26T20:21:49.888957+0000","last_active":"2024-04-26T20:21:51.316932+0000","last_peered":"2024-04-26T20:21:51.316932+0000","last_clean":"2024-04-26T20:21:51.316932+0000","last_became_active":"2024-04-26T20:21:49.888621+0000","last_became_peered":"2024-04-26T20:21:49.888621+0000","last_unstale":"2024-04-26T20:21:51.316932+0000","last_undegraded":"2024-04-26T20:21:51.316932+0000","last_fullsized":"2024-04-26T20:21:51.316932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:53.734129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891334+0000","last_change":"2024-04-26T20:21:48.878830+0000","last_active":"2024-04-26T20:21:50.891334+0000","last_peered":"2024-04-26T20:21:50.891334+0000","last_clean":"2024-04-26T20:21:50.891334+0000","last_became_active":"2024-04-26T20:21:48.878560+0000","last_became_peered":"2024-04-26T20:21:48.878560+0000","last_unstale":"2024-04-26T20:21:50.891334+0000","last_undegraded":"2024-04-26T20:21:50.891334+0000","last_fullsized":"2024-04-26T20:21:50.891334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888017+0000","last_change":"2024-04-26T20:21:49.893919+0000","last_active":"2024-04-26T20:21:50.888017+0000","last_peered":"2024-04-26T20:21:50.888017+0000","last_clean":"2024-04-26T20:21:50.888017+0000","last_became_active":"2024-04-26T20:21:49.893508+0000","last_became_peered":"2024-04-26T20:21:49.893508+0000","last_unstale":"2024-04-26T20:21:50.888017+0000","last_undegraded":"2024-04-26T20:21:50.888017+0000","last_fullsized":"2024-04-26T20:21:50.888017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:42.131574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317700+0000","last_change":"2024-04-26T20:21:48.883061+0000","last_active":"2024-04-26T20:21:51.317700+0000","last_peered":"2024-04-26T20:21:51.317700+0000","last_clean":"2024-04-26T20:21:51.317700+0000","last_became_active":"2024-04-26T20:21:48.882776+0000","last_became_peered":"2024-04-26T20:21:48.882776+0000","last_unstale":"2024-04-26T20:21:51.317700+0000","last_undegraded":"2024-04-26T20:21:51.317700+0000","last_fullsized":"2024-04-26T20:21:51.317700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316756+0000","last_change":"2024-04-26T20:21:49.892633+0000","last_active":"2024-04-26T20:21:51.316756+0000","last_peered":"2024-04-26T20:21:51.316756+0000","last_clean":"2024-04-26T20:21:51.316756+0000","last_became_active":"2024-04-26T20:21:49.892308+0000","last_became_peered":"2024-04-26T20:21:49.892308+0000","last_unstale":"2024-04-26T20:21:51.316756+0000","last_undegraded":"2024-04-26T20:21:51.316756+0000","last_fullsized":"2024-04-26T20:21:51.316756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:59.560594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888824+0000","last_change":"2024-04-26T20:21:48.878718+0000","last_active":"2024-04-26T20:21:50.888824+0000","last_peered":"2024-04-26T20:21:50.888824+0000","last_clean":"2024-04-26T20:21:50.888824+0000","last_became_active":"2024-04-26T20:21:48.878382+0000","last_became_peered":"2024-04-26T20:21:48.878382+0000","last_unstale":"2024-04-26T20:21:50.888824+0000","last_undegraded":"2024-04-26T20:21:50.888824+0000","last_fullsized":"2024-04-26T20:21:50.888824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316423+0000","last_change":"2024-04-26T20:21:49.893414+0000","last_active":"2024-04-26T20:21:51.316423+0000","last_peered":"2024-04-26T20:21:51.316423+0000","last_clean":"2024-04-26T20:21:51.316423+0000","last_became_active":"2024-04-26T20:21:49.893094+0000","last_became_peered":"2024-04-26T20:21:49.893094+0000","last_unstale":"2024-04-26T20:21:51.316423+0000","last_undegraded":"2024-04-26T20:21:51.316423+0000","last_fullsized":"2024-04-26T20:21:51.316423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888774+0000","last_change":"2024-04-26T20:21:48.874766+0000","last_active":"2024-04-26T20:21:50.888774+0000","last_peered":"2024-04-26T20:21:50.888774+0000","last_clean":"2024-04-26T20:21:50.888774+0000","last_became_active":"2024-04-26T20:21:48.874339+0000","last_became_peered":"2024-04-26T20:21:48.874339+0000","last_unstale":"2024-04-26T20:21:50.888774+0000","last_undegraded":"2024-04-26T20:21:50.888774+0000","last_fullsized":"2024-04-26T20:21:50.888774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316594+0000","last_change":"2024-04-26T20:21:49.893713+0000","last_active":"2024-04-26T20:21:51.316594+0000","last_peered":"2024-04-26T20:21:51.316594+0000","last_clean":"2024-04-26T20:21:51.316594+0000","last_became_active":"2024-04-26T20:21:49.891376+0000","last_became_peered":"2024-04-26T20:21:49.891376+0000","last_unstale":"2024-04-26T20:21:51.316594+0000","last_undegraded":"2024-04-26T20:21:51.316594+0000","last_fullsized":"2024-04-26T20:21:51.316594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318283+0000","last_change":"2024-04-26T20:21:48.882680+0000","last_active":"2024-04-26T20:21:51.318283+0000","last_peered":"2024-04-26T20:21:51.318283+0000","last_clean":"2024-04-26T20:21:51.318283+0000","last_became_active":"2024-04-26T20:21:48.882393+0000","last_became_peered":"2024-04-26T20:21:48.882393+0000","last_unstale":"2024-04-26T20:21:51.318283+0000","last_undegraded":"2024-04-26T20:21:51.318283+0000","last_fullsized":"2024-04-26T20:21:51.318283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:41.422336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891134+0000","last_change":"2024-04-26T20:21:49.892761+0000","last_active":"2024-04-26T20:21:50.891134+0000","last_peered":"2024-04-26T20:21:50.891134+0000","last_clean":"2024-04-26T20:21:50.891134+0000","last_became_active":"2024-04-26T20:21:49.892403+0000","last_became_peered":"2024-04-26T20:21:49.892403+0000","last_unstale":"2024-04-26T20:21:50.891134+0000","last_undegraded":"2024-04-26T20:21:50.891134+0000","last_fullsized":"2024-04-26T20:21:50.891134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317815+0000","last_change":"2024-04-26T20:21:49.890488+0000","last_active":"2024-04-26T20:21:51.317815+0000","last_peered":"2024-04-26T20:21:51.317815+0000","last_clean":"2024-04-26T20:21:51.317815+0000","last_became_active":"2024-04-26T20:21:49.889950+0000","last_became_peered":"2024-04-26T20:21:49.889950+0000","last_unstale":"2024-04-26T20:21:51.317815+0000","last_undegraded":"2024-04-26T20:21:51.317815+0000","last_fullsized":"2024-04-26T20:21:51.317815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317232+0000","last_change":"2024-04-26T20:21:48.884202+0000","last_active":"2024-04-26T20:21:51.317232+0000","last_peered":"2024-04-26T20:21:51.317232+0000","last_clean":"2024-04-26T20:21:51.317232+0000","last_became_active":"2024-04-26T20:21:48.883945+0000","last_became_peered":"2024-04-26T20:21:48.883945+0000","last_unstale":"2024-04-26T20:21:51.317232+0000","last_undegraded":"2024-04-26T20:21:51.317232+0000","last_fullsized":"2024-04-26T20:21:51.317232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:25.685618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318377+0000","last_change":"2024-04-26T20:21:49.894650+0000","last_active":"2024-04-26T20:21:51.318377+0000","last_peered":"2024-04-26T20:21:51.318377+0000","last_clean":"2024-04-26T20:21:51.318377+0000","last_became_active":"2024-04-26T20:21:49.889860+0000","last_became_peered":"2024-04-26T20:21:49.889860+0000","last_unstale":"2024-04-26T20:21:51.318377+0000","last_undegraded":"2024-04-26T20:21:51.318377+0000","last_fullsized":"2024-04-26T20:21:51.318377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:13.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317454+0000","last_change":"2024-04-26T20:21:48.881647+0000","last_active":"2024-04-26T20:21:51.317454+0000","last_peered":"2024-04-26T20:21:51.317454+0000","last_clean":"2024-04-26T20:21:51.317454+0000","last_became_active":"2024-04-26T20:21:48.881349+0000","last_became_peered":"2024-04-26T20:21:48.881349+0000","last_unstale":"2024-04-26T20:21:51.317454+0000","last_undegraded":"2024-04-26T20:21:51.317454+0000","last_fullsized":"2024-04-26T20:21:51.317454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:24.800032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888111+0000","last_change":"2024-04-26T20:21:49.884044+0000","last_active":"2024-04-26T20:21:50.888111+0000","last_peered":"2024-04-26T20:21:50.888111+0000","last_clean":"2024-04-26T20:21:50.888111+0000","last_became_active":"2024-04-26T20:21:49.883750+0000","last_became_peered":"2024-04-26T20:21:49.883750+0000","last_unstale":"2024-04-26T20:21:50.888111+0000","last_undegraded":"2024-04-26T20:21:50.888111+0000","last_fullsized":"2024-04-26T20:21:50.888111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316858+0000","last_change":"2024-04-26T20:21:48.875662+0000","last_active":"2024-04-26T20:21:51.316858+0000","last_peered":"2024-04-26T20:21:51.316858+0000","last_clean":"2024-04-26T20:21:51.316858+0000","last_became_active":"2024-04-26T20:21:48.875024+0000","last_became_peered":"2024-04-26T20:21:48.875024+0000","last_unstale":"2024-04-26T20:21:51.316858+0000","last_undegraded":"2024-04-26T20:21:51.316858+0000","last_fullsized":"2024-04-26T20:21:51.316858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:40:48.030416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316941+0000","last_change":"2024-04-26T20:21:49.894476+0000","last_active":"2024-04-26T20:21:51.316941+0000","last_peered":"2024-04-26T20:21:51.316941+0000","last_clean":"2024-04-26T20:21:51.316941+0000","last_became_active":"2024-04-26T20:21:49.894186+0000","last_became_peered":"2024-04-26T20:21:49.894186+0000","last_unstale":"2024-04-26T20:21:51.316941+0000","last_undegraded":"2024-04-26T20:21:51.316941+0000","last_fullsized":"2024-04-26T20:21:51.316941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:27.648550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888239+0000","last_change":"2024-04-26T20:21:48.880562+0000","last_active":"2024-04-26T20:21:50.888239+0000","last_peered":"2024-04-26T20:21:50.888239+0000","last_clean":"2024-04-26T20:21:50.888239+0000","last_became_active":"2024-04-26T20:21:48.880249+0000","last_became_peered":"2024-04-26T20:21:48.880249+0000","last_unstale":"2024-04-26T20:21:50.888239+0000","last_undegraded":"2024-04-26T20:21:50.888239+0000","last_fullsized":"2024-04-26T20:21:50.888239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:21:53.902 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:21:54.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.037+0000 7fe087669640 1 Processor -- start 2024-04-26T20:21:54.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.037+0000 7fe087669640 1 -- start start 2024-04-26T20:21:54.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.038+0000 7fe087669640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe08015f380 0x7fe0800d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.038+0000 7fe087669640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0800d3de0 con 0x7fe0800b9a90 2024-04-26T20:21:54.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.038+0000 7fe087669640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0800d40c0 con 0x7fe08015f760 2024-04-26T20:21:54.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.038+0000 7fe087669640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fe0800d43a0 con 0x7fe08015f380 2024-04-26T20:21:54.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.038+0000 7fe085405640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe08015f380 0x7fe0800d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.038+0000 7fe085c06640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe0800b9a90 0x7fe0800d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52730/0 (socket says 172.21.15.37:52730) 2024-04-26T20:21:54.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.038+0000 7fe085c06640 1 -- 172.21.15.37:0/3011439538 learned_addr learned my addr 172.21.15.37:0/3011439538 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:54.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.038+0000 7fe085405640 1 --2- 172.21.15.37:0/3011439538 >> v2:172.21.15.37:3300/0 conn(0x7fe08015f380 0x7fe0800d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59530/0 (socket says 172.21.15.37:59530) 2024-04-26T20:21:54.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.039+0000 7fe085405640 1 -- 172.21.15.37:0/3011439538 >> v1:172.21.15.134:6789/0 conn(0x7fe08015f760 legacy=0x7fe0800cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.039+0000 7fe085405640 1 -- 172.21.15.37:0/3011439538 >> v1:172.21.15.37:6789/0 conn(0x7fe0800b9a90 legacy=0x7fe0800d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.039+0000 7fe085405640 1 -- 172.21.15.37:0/3011439538 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0800b8280 con 0x7fe08015f380 2024-04-26T20:21:54.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.039+0000 7fe07ffff640 1 -- 172.21.15.37:0/3011439538 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1613675666 0 0) 0x7fe0800d3de0 con 0x7fe0800b9a90 2024-04-26T20:21:54.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.039+0000 7fe085405640 1 --2- 172.21.15.37:0/3011439538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08015f380 0x7fe0800d1bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0780097a0 tx=0x7fe07802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a6d3e0eea8246e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:54.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.039+0000 7fe07ffff640 1 -- 172.21.15.37:0/3011439538 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078037070 con 0x7fe08015f380 2024-04-26T20:21:54.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.039+0000 7fe07ffff640 1 -- 172.21.15.37:0/3011439538 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe078032640 con 0x7fe08015f380 2024-04-26T20:21:54.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.039+0000 7fe07ffff640 1 -- 172.21.15.37:0/3011439538 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078032a10 con 0x7fe08015f380 2024-04-26T20:21:54.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.040+0000 7fe087669640 1 -- 172.21.15.37:0/3011439538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08015f380 msgr2=0x7fe0800d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.040+0000 7fe087669640 1 --2- 172.21.15.37:0/3011439538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08015f380 0x7fe0800d1bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0780097a0 tx=0x7fe07802a2c0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.040+0000 7fe087669640 1 -- 172.21.15.37:0/3011439538 shutdown_connections 2024-04-26T20:21:54.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.040+0000 7fe087669640 1 --2- 172.21.15.37:0/3011439538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08015f380 0x7fe0800d1bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.040+0000 7fe087669640 1 -- 172.21.15.37:0/3011439538 >> 172.21.15.37:0/3011439538 conn(0x7fe08009e560 msgr2=0x7fe0800d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:54.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.040+0000 7fe087669640 1 -- 172.21.15.37:0/3011439538 shutdown_connections 2024-04-26T20:21:54.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.040+0000 7fe087669640 1 -- 172.21.15.37:0/3011439538 wait complete. 2024-04-26T20:21:54.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.041+0000 7fe087669640 1 Processor -- start 2024-04-26T20:21:54.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.041+0000 7fe087669640 1 -- start start 2024-04-26T20:21:54.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe087669640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0800b9a90 0x7fe08015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe087669640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08015f380 0x7fe08015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe087669640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe08015f760 0x7fe0800cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe087669640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0800d1100 con 0x7fe08015f380 2024-04-26T20:21:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe087669640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0800d0cc0 con 0x7fe0800b9a90 2024-04-26T20:21:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe087669640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe0800d1280 con 0x7fe08015f760 2024-04-26T20:21:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe084c04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08015f380 0x7fe08015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe084c04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08015f380 0x7fe08015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59544/0 (socket says 172.21.15.37:59544) 2024-04-26T20:21:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe084c04640 1 -- 172.21.15.37:0/722648499 learned_addr learned my addr 172.21.15.37:0/722648499 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe085405640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0800b9a90 0x7fe08015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe085c06640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe08015f760 0x7fe0800cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe084c04640 1 -- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe08015f760 msgr2=0x7fe0800cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe084c04640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe08015f760 0x7fe0800cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe084c04640 1 -- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0800b9a90 msgr2=0x7fe08015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe084c04640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0800b9a90 0x7fe08015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.042+0000 7fe084c04640 1 -- 172.21.15.37:0/722648499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08015da00 con 0x7fe08015f380 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.043+0000 7fe084c04640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08015f380 0x7fe08015d120 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c440 tx=0x7fe074001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.043+0000 7fe07dffb640 1 -- 172.21.15.37:0/722648499 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0740052d0 con 0x7fe08015f380 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.043+0000 7fe07dffb640 1 -- 172.21.15.37:0/722648499 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe074005470 con 0x7fe08015f380 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.043+0000 7fe087669640 1 -- 172.21.15.37:0/722648499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0800cd800 con 0x7fe08015f380 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.043+0000 7fe07dffb640 1 -- 172.21.15.37:0/722648499 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074007260 con 0x7fe08015f380 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.043+0000 7fe087669640 1 -- 172.21.15.37:0/722648499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08015b2d0 con 0x7fe08015f380 2024-04-26T20:21:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.044+0000 7fe087669640 1 -- 172.21.15.37:0/722648499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe08006a360 con 0x7fe08015f380 2024-04-26T20:21:54.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.045+0000 7fe07dffb640 1 -- 172.21.15.37:0/722648499 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe074007400 con 0x7fe08015f380 2024-04-26T20:21:54.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.045+0000 7fe07dffb640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe06c06c8f0 0x7fe06c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.045+0000 7fe07dffb640 1 -- 172.21.15.37:0/722648499 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe07408afc0 con 0x7fe08015f380 2024-04-26T20:21:54.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.045+0000 7fe085405640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe06c06c8f0 0x7fe06c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.046+0000 7fe085405640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe06c06c8f0 0x7fe06c06edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe078034000 tx=0x7fe07802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:54.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.049+0000 7fe07dffb640 1 -- 172.21.15.37:0/722648499 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe074058da0 con 0x7fe08015f380 2024-04-26T20:21:54.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.229+0000 7fe087669640 1 -- 172.21.15.37:0/722648499 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0800b6f40 con 0x7fe06c06c8f0 2024-04-26T20:21:54.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.236+0000 7fe07dffb640 1 -- 172.21.15.37:0/722648499 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7fe0800b6f40 con 0x7fe06c06c8f0 2024-04-26T20:21:54.239 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:54.240 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:21:54.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 -- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe06c06c8f0 msgr2=0x7fe06c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe06c06c8f0 0x7fe06c06edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe078034000 tx=0x7fe07802e000 comp rx=0 tx=0).stop 2024-04-26T20:21:54.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 -- 172.21.15.37:0/722648499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08015f380 msgr2=0x7fe08015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08015f380 0x7fe08015d120 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c440 tx=0x7fe074001e70 comp rx=0 tx=0).stop 2024-04-26T20:21:54.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 -- 172.21.15.37:0/722648499 shutdown_connections 2024-04-26T20:21:54.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe06c06c8f0 0x7fe06c06edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe08015f760 0x7fe0800cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08015f380 0x7fe08015d120 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 --2- 172.21.15.37:0/722648499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0800b9a90 0x7fe08015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 -- 172.21.15.37:0/722648499 >> 172.21.15.37:0/722648499 conn(0x7fe08009e560 msgr2=0x7fe08006d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:54.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 -- 172.21.15.37:0/722648499 shutdown_connections 2024-04-26T20:21:54.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.238+0000 7fe087669640 1 -- 172.21.15.37:0/722648499 wait complete. 2024-04-26T20:21:54.250 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-26T20:21:53.335032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.478384"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317460+0000","last_change":"2024-04-26T20:21:49.899185+0000","last_active":"2024-04-26T20:21:51.317460+0000","last_peered":"2024-04-26T20:21:51.317460+0000","last_clean":"2024-04-26T20:21:51.317460+0000","last_became_active":"2024-04-26T20:21:49.898818+0000","last_became_peered":"2024-04-26T20:21:49.898818+0000","last_unstale":"2024-04-26T20:21:51.317460+0000","last_undegraded":"2024-04-26T20:21:51.317460+0000","last_fullsized":"2024-04-26T20:21:51.317460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:13.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.889053+0000","last_change":"2024-04-26T20:21:49.899484+0000","last_active":"2024-04-26T20:21:50.889053+0000","last_peered":"2024-04-26T20:21:50.889053+0000","last_clean":"2024-04-26T20:21:50.889053+0000","last_became_active":"2024-04-26T20:21:49.899091+0000","last_became_peered":"2024-04-26T20:21:49.899091+0000","last_unstale":"2024-04-26T20:21:50.889053+0000","last_undegraded":"2024-04-26T20:21:50.889053+0000","last_fullsized":"2024-04-26T20:21:50.889053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.887979+0000","last_change":"2024-04-26T20:21:49.881814+0000","last_active":"2024-04-26T20:21:50.887979+0000","last_peered":"2024-04-26T20:21:50.887979+0000","last_clean":"2024-04-26T20:21:50.887979+0000","last_became_active":"2024-04-26T20:21:49.881507+0000","last_became_peered":"2024-04-26T20:21:49.881507+0000","last_unstale":"2024-04-26T20:21:50.887979+0000","last_undegraded":"2024-04-26T20:21:50.887979+0000","last_fullsized":"2024-04-26T20:21:50.887979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:30.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318197+0000","last_change":"2024-04-26T20:21:49.896399+0000","last_active":"2024-04-26T20:21:51.318197+0000","last_peered":"2024-04-26T20:21:51.318197+0000","last_clean":"2024-04-26T20:21:51.318197+0000","last_became_active":"2024-04-26T20:21:49.896136+0000","last_became_peered":"2024-04-26T20:21:49.896136+0000","last_unstale":"2024-04-26T20:21:51.318197+0000","last_undegraded":"2024-04-26T20:21:51.318197+0000","last_fullsized":"2024-04-26T20:21:51.318197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.737636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317925+0000","last_change":"2024-04-26T20:21:49.898821+0000","last_active":"2024-04-26T20:21:51.317925+0000","last_peered":"2024-04-26T20:21:51.317925+0000","last_clean":"2024-04-26T20:21:51.317925+0000","last_became_active":"2024-04-26T20:21:49.898554+0000","last_became_peered":"2024-04-26T20:21:49.898554+0000","last_unstale":"2024-04-26T20:21:51.317925+0000","last_undegraded":"2024-04-26T20:21:51.317925+0000","last_fullsized":"2024-04-26T20:21:51.317925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:52.618628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316822+0000","last_change":"2024-04-26T20:21:49.878509+0000","last_active":"2024-04-26T20:21:51.316822+0000","last_peered":"2024-04-26T20:21:51.316822+0000","last_clean":"2024-04-26T20:21:51.316822+0000","last_became_active":"2024-04-26T20:21:49.878237+0000","last_became_peered":"2024-04-26T20:21:49.878237+0000","last_unstale":"2024-04-26T20:21:51.316822+0000","last_undegraded":"2024-04-26T20:21:51.316822+0000","last_fullsized":"2024-04-26T20:21:51.316822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316684+0000","last_change":"2024-04-26T20:21:49.898211+0000","last_active":"2024-04-26T20:21:51.316684+0000","last_peered":"2024-04-26T20:21:51.316684+0000","last_clean":"2024-04-26T20:21:51.316684+0000","last_became_active":"2024-04-26T20:21:49.897952+0000","last_became_peered":"2024-04-26T20:21:49.897952+0000","last_unstale":"2024-04-26T20:21:51.316684+0000","last_undegraded":"2024-04-26T20:21:51.316684+0000","last_fullsized":"2024-04-26T20:21:51.316684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316222+0000","last_change":"2024-04-26T20:21:49.892897+0000","last_active":"2024-04-26T20:21:51.316222+0000","last_peered":"2024-04-26T20:21:51.316222+0000","last_clean":"2024-04-26T20:21:51.316222+0000","last_became_active":"2024-04-26T20:21:49.890640+0000","last_became_peered":"2024-04-26T20:21:49.890640+0000","last_unstale":"2024-04-26T20:21:51.316222+0000","last_undegraded":"2024-04-26T20:21:51.316222+0000","last_fullsized":"2024-04-26T20:21:51.316222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:42.131574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317597+0000","last_change":"2024-04-26T20:21:49.901710+0000","last_active":"2024-04-26T20:21:51.317597+0000","last_peered":"2024-04-26T20:21:51.317597+0000","last_clean":"2024-04-26T20:21:51.317597+0000","last_became_active":"2024-04-26T20:21:49.901363+0000","last_became_peered":"2024-04-26T20:21:49.901363+0000","last_unstale":"2024-04-26T20:21:51.317597+0000","last_undegraded":"2024-04-26T20:21:51.317597+0000","last_fullsized":"2024-04-26T20:21:51.317597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:50.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891437+0000","last_change":"2024-04-26T20:21:49.900636+0000","last_active":"2024-04-26T20:21:50.891437+0000","last_peered":"2024-04-26T20:21:50.891437+0000","last_clean":"2024-04-26T20:21:50.891437+0000","last_became_active":"2024-04-26T20:21:49.900073+0000","last_became_peered":"2024-04-26T20:21:49.900073+0000","last_unstale":"2024-04-26T20:21:50.891437+0000","last_undegraded":"2024-04-26T20:21:50.891437+0000","last_fullsized":"2024-04-26T20:21:50.891437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891162+0000","last_change":"2024-04-26T20:21:49.892413+0000","last_active":"2024-04-26T20:21:50.891162+0000","last_peered":"2024-04-26T20:21:50.891162+0000","last_clean":"2024-04-26T20:21:50.891162+0000","last_became_active":"2024-04-26T20:21:49.892106+0000","last_became_peered":"2024-04-26T20:21:49.892106+0000","last_unstale":"2024-04-26T20:21:50.891162+0000","last_undegraded":"2024-04-26T20:21:50.891162+0000","last_fullsized":"2024-04-26T20:21:50.891162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317031+0000","last_change":"2024-04-26T20:21:49.894073+0000","last_active":"2024-04-26T20:21:51.317031+0000","last_peered":"2024-04-26T20:21:51.317031+0000","last_clean":"2024-04-26T20:21:51.317031+0000","last_became_active":"2024-04-26T20:21:49.893726+0000","last_became_peered":"2024-04-26T20:21:49.893726+0000","last_unstale":"2024-04-26T20:21:51.317031+0000","last_undegraded":"2024-04-26T20:21:51.317031+0000","last_fullsized":"2024-04-26T20:21:51.317031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:00.300212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316742+0000","last_change":"2024-04-26T20:21:49.895923+0000","last_active":"2024-04-26T20:21:51.316742+0000","last_peered":"2024-04-26T20:21:51.316742+0000","last_clean":"2024-04-26T20:21:51.316742+0000","last_became_active":"2024-04-26T20:21:49.895261+0000","last_became_peered":"2024-04-26T20:21:49.895261+0000","last_unstale":"2024-04-26T20:21:51.316742+0000","last_undegraded":"2024-04-26T20:21:51.316742+0000","last_fullsized":"2024-04-26T20:21:51.316742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.887942+0000","last_change":"2024-04-26T20:21:49.904360+0000","last_active":"2024-04-26T20:21:50.887942+0000","last_peered":"2024-04-26T20:21:50.887942+0000","last_clean":"2024-04-26T20:21:50.887942+0000","last_became_active":"2024-04-26T20:21:49.899936+0000","last_became_peered":"2024-04-26T20:21:49.899936+0000","last_unstale":"2024-04-26T20:21:50.887942+0000","last_undegraded":"2024-04-26T20:21:50.887942+0000","last_fullsized":"2024-04-26T20:21:50.887942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:55.987722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888860+0000","last_change":"2024-04-26T20:21:49.893445+0000","last_active":"2024-04-26T20:21:50.888860+0000","last_peered":"2024-04-26T20:21:50.888860+0000","last_clean":"2024-04-26T20:21:50.888860+0000","last_became_active":"2024-04-26T20:21:49.892704+0000","last_became_peered":"2024-04-26T20:21:49.892704+0000","last_unstale":"2024-04-26T20:21:50.888860+0000","last_undegraded":"2024-04-26T20:21:50.888860+0000","last_fullsized":"2024-04-26T20:21:50.888860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316915+0000","last_change":"2024-04-26T20:21:49.883172+0000","last_active":"2024-04-26T20:21:51.316915+0000","last_peered":"2024-04-26T20:21:51.316915+0000","last_clean":"2024-04-26T20:21:51.316915+0000","last_became_active":"2024-04-26T20:21:49.882916+0000","last_became_peered":"2024-04-26T20:21:49.882916+0000","last_unstale":"2024-04-26T20:21:51.316915+0000","last_undegraded":"2024-04-26T20:21:51.316915+0000","last_fullsized":"2024-04-26T20:21:51.316915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317389+0000","last_change":"2024-04-26T20:21:49.889742+0000","last_active":"2024-04-26T20:21:51.317389+0000","last_peered":"2024-04-26T20:21:51.317389+0000","last_clean":"2024-04-26T20:21:51.317389+0000","last_became_active":"2024-04-26T20:21:49.889417+0000","last_became_peered":"2024-04-26T20:21:49.889417+0000","last_unstale":"2024-04-26T20:21:51.317389+0000","last_undegraded":"2024-04-26T20:21:51.317389+0000","last_fullsized":"2024-04-26T20:21:51.317389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:53.734129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890942+0000","last_change":"2024-04-26T20:21:49.894387+0000","last_active":"2024-04-26T20:21:50.890942+0000","last_peered":"2024-04-26T20:21:50.890942+0000","last_clean":"2024-04-26T20:21:50.890942+0000","last_became_active":"2024-04-26T20:21:49.893464+0000","last_became_peered":"2024-04-26T20:21:49.893464+0000","last_unstale":"2024-04-26T20:21:50.890942+0000","last_undegraded":"2024-04-26T20:21:50.890942+0000","last_fullsized":"2024-04-26T20:21:50.890942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:26.689749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317646+0000","last_change":"2024-04-26T20:21:49.892977+0000","last_active":"2024-04-26T20:21:51.317646+0000","last_peered":"2024-04-26T20:21:51.317646+0000","last_clean":"2024-04-26T20:21:51.317646+0000","last_became_active":"2024-04-26T20:21:49.892709+0000","last_became_peered":"2024-04-26T20:21:49.892709+0000","last_unstale":"2024-04-26T20:21:51.317646+0000","last_undegraded":"2024-04-26T20:21:51.317646+0000","last_fullsized":"2024-04-26T20:21:51.317646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:44.689592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891273+0000","last_change":"2024-04-26T20:21:49.892717+0000","last_active":"2024-04-26T20:21:50.891273+0000","last_peered":"2024-04-26T20:21:50.891273+0000","last_clean":"2024-04-26T20:21:50.891273+0000","last_became_active":"2024-04-26T20:21:49.892029+0000","last_became_peered":"2024-04-26T20:21:49.892029+0000","last_unstale":"2024-04-26T20:21:50.891273+0000","last_undegraded":"2024-04-26T20:21:50.891273+0000","last_fullsized":"2024-04-26T20:21:50.891273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888152+0000","last_change":"2024-04-26T20:21:49.891937+0000","last_active":"2024-04-26T20:21:50.888152+0000","last_peered":"2024-04-26T20:21:50.888152+0000","last_clean":"2024-04-26T20:21:50.888152+0000","last_became_active":"2024-04-26T20:21:49.891631+0000","last_became_peered":"2024-04-26T20:21:49.891631+0000","last_unstale":"2024-04-26T20:21:50.888152+0000","last_undegraded":"2024-04-26T20:21:50.888152+0000","last_fullsized":"2024-04-26T20:21:50.888152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318190+0000","last_change":"2024-04-26T20:21:49.890285+0000","last_active":"2024-04-26T20:21:51.318190+0000","last_peered":"2024-04-26T20:21:51.318190+0000","last_clean":"2024-04-26T20:21:51.318190+0000","last_became_active":"2024-04-26T20:21:49.888800+0000","last_became_peered":"2024-04-26T20:21:49.888800+0000","last_unstale":"2024-04-26T20:21:51.318190+0000","last_undegraded":"2024-04-26T20:21:51.318190+0000","last_fullsized":"2024-04-26T20:21:51.318190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:32.476892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316465+0000","last_change":"2024-04-26T20:21:49.890345+0000","last_active":"2024-04-26T20:21:51.316465+0000","last_peered":"2024-04-26T20:21:51.316465+0000","last_clean":"2024-04-26T20:21:51.316465+0000","last_became_active":"2024-04-26T20:21:49.889999+0000","last_became_peered":"2024-04-26T20:21:49.889999+0000","last_unstale":"2024-04-26T20:21:51.316465+0000","last_undegraded":"2024-04-26T20:21:51.316465+0000","last_fullsized":"2024-04-26T20:21:51.316465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317064+0000","last_change":"2024-04-26T20:21:49.897497+0000","last_active":"2024-04-26T20:21:51.317064+0000","last_peered":"2024-04-26T20:21:51.317064+0000","last_clean":"2024-04-26T20:21:51.317064+0000","last_became_active":"2024-04-26T20:21:49.896968+0000","last_became_peered":"2024-04-26T20:21:49.896968+0000","last_unstale":"2024-04-26T20:21:51.317064+0000","last_undegraded":"2024-04-26T20:21:51.317064+0000","last_fullsized":"2024-04-26T20:21:51.317064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316953+0000","last_change":"2024-04-26T20:21:49.890916+0000","last_active":"2024-04-26T20:21:51.316953+0000","last_peered":"2024-04-26T20:21:51.316953+0000","last_clean":"2024-04-26T20:21:51.316953+0000","last_became_active":"2024-04-26T20:21:49.890517+0000","last_became_peered":"2024-04-26T20:21:49.890517+0000","last_unstale":"2024-04-26T20:21:51.316953+0000","last_undegraded":"2024-04-26T20:21:51.316953+0000","last_fullsized":"2024-04-26T20:21:51.316953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891447+0000","last_change":"2024-04-26T20:21:49.903237+0000","last_active":"2024-04-26T20:21:50.891447+0000","last_peered":"2024-04-26T20:21:50.891447+0000","last_clean":"2024-04-26T20:21:50.891447+0000","last_became_active":"2024-04-26T20:21:49.902924+0000","last_became_peered":"2024-04-26T20:21:49.902924+0000","last_unstale":"2024-04-26T20:21:50.891447+0000","last_undegraded":"2024-04-26T20:21:50.891447+0000","last_fullsized":"2024-04-26T20:21:50.891447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888809+0000","last_change":"2024-04-26T20:21:49.905100+0000","last_active":"2024-04-26T20:21:50.888809+0000","last_peered":"2024-04-26T20:21:50.888809+0000","last_clean":"2024-04-26T20:21:50.888809+0000","last_became_active":"2024-04-26T20:21:49.904037+0000","last_became_peered":"2024-04-26T20:21:49.904037+0000","last_unstale":"2024-04-26T20:21:50.888809+0000","last_undegraded":"2024-04-26T20:21:50.888809+0000","last_fullsized":"2024-04-26T20:21:50.888809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316677+0000","last_change":"2024-04-26T20:21:49.891939+0000","last_active":"2024-04-26T20:21:51.316677+0000","last_peered":"2024-04-26T20:21:51.316677+0000","last_clean":"2024-04-26T20:21:51.316677+0000","last_became_active":"2024-04-26T20:21:49.891662+0000","last_became_peered":"2024-04-26T20:21:49.891662+0000","last_unstale":"2024-04-26T20:21:51.316677+0000","last_undegraded":"2024-04-26T20:21:51.316677+0000","last_fullsized":"2024-04-26T20:21:51.316677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888299+0000","last_change":"2024-04-26T20:21:49.895730+0000","last_active":"2024-04-26T20:21:50.888299+0000","last_peered":"2024-04-26T20:21:50.888299+0000","last_clean":"2024-04-26T20:21:50.888299+0000","last_became_active":"2024-04-26T20:21:49.895244+0000","last_became_peered":"2024-04-26T20:21:49.895244+0000","last_unstale":"2024-04-26T20:21:50.888299+0000","last_undegraded":"2024-04-26T20:21:50.888299+0000","last_fullsized":"2024-04-26T20:21:50.888299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318438+0000","last_change":"2024-04-26T20:21:49.896367+0000","last_active":"2024-04-26T20:21:51.318438+0000","last_peered":"2024-04-26T20:21:51.318438+0000","last_clean":"2024-04-26T20:21:51.318438+0000","last_became_active":"2024-04-26T20:21:49.895963+0000","last_became_peered":"2024-04-26T20:21:49.895963+0000","last_unstale":"2024-04-26T20:21:51.318438+0000","last_undegraded":"2024-04-26T20:21:51.318438+0000","last_fullsized":"2024-04-26T20:21:51.318438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:57.212796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890871+0000","last_change":"2024-04-26T20:21:49.880643+0000","last_active":"2024-04-26T20:21:50.890871+0000","last_peered":"2024-04-26T20:21:50.890871+0000","last_clean":"2024-04-26T20:21:50.890871+0000","last_became_active":"2024-04-26T20:21:49.880295+0000","last_became_peered":"2024-04-26T20:21:49.880295+0000","last_unstale":"2024-04-26T20:21:50.890871+0000","last_undegraded":"2024-04-26T20:21:50.890871+0000","last_fullsized":"2024-04-26T20:21:50.890871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:40.680121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891215+0000","last_change":"2024-04-26T20:21:49.906308+0000","last_active":"2024-04-26T20:21:50.891215+0000","last_peered":"2024-04-26T20:21:50.891215+0000","last_clean":"2024-04-26T20:21:50.891215+0000","last_became_active":"2024-04-26T20:21:49.906092+0000","last_became_peered":"2024-04-26T20:21:49.906092+0000","last_unstale":"2024-04-26T20:21:50.891215+0000","last_undegraded":"2024-04-26T20:21:50.891215+0000","last_fullsized":"2024-04-26T20:21:50.891215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318011+0000","last_change":"2024-04-26T20:21:49.901086+0000","last_active":"2024-04-26T20:21:51.318011+0000","last_peered":"2024-04-26T20:21:51.318011+0000","last_clean":"2024-04-26T20:21:51.318011+0000","last_became_active":"2024-04-26T20:21:49.900797+0000","last_became_peered":"2024-04-26T20:21:49.900797+0000","last_unstale":"2024-04-26T20:21:51.318011+0000","last_undegraded":"2024-04-26T20:21:51.318011+0000","last_fullsized":"2024-04-26T20:21:51.318011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:26.689749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316705+0000","last_change":"2024-04-26T20:21:48.880856+0000","last_active":"2024-04-26T20:21:51.316705+0000","last_peered":"2024-04-26T20:21:51.316705+0000","last_clean":"2024-04-26T20:21:51.316705+0000","last_became_active":"2024-04-26T20:21:48.880512+0000","last_became_peered":"2024-04-26T20:21:48.880512+0000","last_unstale":"2024-04-26T20:21:51.316705+0000","last_undegraded":"2024-04-26T20:21:51.316705+0000","last_fullsized":"2024-04-26T20:21:51.316705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888381+0000","last_change":"2024-04-26T20:21:49.889888+0000","last_active":"2024-04-26T20:21:50.888381+0000","last_peered":"2024-04-26T20:21:50.888381+0000","last_clean":"2024-04-26T20:21:50.888381+0000","last_became_active":"2024-04-26T20:21:49.889451+0000","last_became_peered":"2024-04-26T20:21:49.889451+0000","last_unstale":"2024-04-26T20:21:50.888381+0000","last_undegraded":"2024-04-26T20:21:50.888381+0000","last_fullsized":"2024-04-26T20:21:50.888381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891210+0000","last_change":"2024-04-26T20:21:48.880648+0000","last_active":"2024-04-26T20:21:50.891210+0000","last_peered":"2024-04-26T20:21:50.891210+0000","last_clean":"2024-04-26T20:21:50.891210+0000","last_became_active":"2024-04-26T20:21:48.880389+0000","last_became_peered":"2024-04-26T20:21:48.880389+0000","last_unstale":"2024-04-26T20:21:50.891210+0000","last_undegraded":"2024-04-26T20:21:50.891210+0000","last_fullsized":"2024-04-26T20:21:50.891210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316415+0000","last_change":"2024-04-26T20:21:48.873199+0000","last_active":"2024-04-26T20:21:51.316415+0000","last_peered":"2024-04-26T20:21:51.316415+0000","last_clean":"2024-04-26T20:21:51.316415+0000","last_became_active":"2024-04-26T20:21:48.872868+0000","last_became_peered":"2024-04-26T20:21:48.872868+0000","last_unstale":"2024-04-26T20:21:51.316415+0000","last_undegraded":"2024-04-26T20:21:51.316415+0000","last_fullsized":"2024-04-26T20:21:51.316415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316315+0000","last_change":"2024-04-26T20:21:49.897450+0000","last_active":"2024-04-26T20:21:51.316315+0000","last_peered":"2024-04-26T20:21:51.316315+0000","last_clean":"2024-04-26T20:21:51.316315+0000","last_became_active":"2024-04-26T20:21:49.897165+0000","last_became_peered":"2024-04-26T20:21:49.897165+0000","last_unstale":"2024-04-26T20:21:51.316315+0000","last_undegraded":"2024-04-26T20:21:51.316315+0000","last_fullsized":"2024-04-26T20:21:51.316315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317170+0000","last_change":"2024-04-26T20:21:48.881023+0000","last_active":"2024-04-26T20:21:51.317170+0000","last_peered":"2024-04-26T20:21:51.317170+0000","last_clean":"2024-04-26T20:21:51.317170+0000","last_became_active":"2024-04-26T20:21:48.880610+0000","last_became_peered":"2024-04-26T20:21:48.880610+0000","last_unstale":"2024-04-26T20:21:51.317170+0000","last_undegraded":"2024-04-26T20:21:51.317170+0000","last_fullsized":"2024-04-26T20:21:51.317170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888715+0000","last_change":"2024-04-26T20:21:49.905734+0000","last_active":"2024-04-26T20:21:50.888715+0000","last_peered":"2024-04-26T20:21:50.888715+0000","last_clean":"2024-04-26T20:21:50.888715+0000","last_became_active":"2024-04-26T20:21:49.905424+0000","last_became_peered":"2024-04-26T20:21:49.905424+0000","last_unstale":"2024-04-26T20:21:50.888715+0000","last_undegraded":"2024-04-26T20:21:50.888715+0000","last_fullsized":"2024-04-26T20:21:50.888715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891379+0000","last_change":"2024-04-26T20:21:48.877043+0000","last_active":"2024-04-26T20:21:50.891379+0000","last_peered":"2024-04-26T20:21:50.891379+0000","last_clean":"2024-04-26T20:21:50.891379+0000","last_became_active":"2024-04-26T20:21:48.876721+0000","last_became_peered":"2024-04-26T20:21:48.876721+0000","last_unstale":"2024-04-26T20:21:50.891379+0000","last_undegraded":"2024-04-26T20:21:50.891379+0000","last_fullsized":"2024-04-26T20:21:50.891379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317752+0000","last_change":"2024-04-26T20:21:49.890323+0000","last_active":"2024-04-26T20:21:51.317752+0000","last_peered":"2024-04-26T20:21:51.317752+0000","last_clean":"2024-04-26T20:21:51.317752+0000","last_became_active":"2024-04-26T20:21:49.889790+0000","last_became_peered":"2024-04-26T20:21:49.889790+0000","last_unstale":"2024-04-26T20:21:51.317752+0000","last_undegraded":"2024-04-26T20:21:51.317752+0000","last_fullsized":"2024-04-26T20:21:51.317752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:27.648550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317863+0000","last_change":"2024-04-26T20:21:48.872759+0000","last_active":"2024-04-26T20:21:51.317863+0000","last_peered":"2024-04-26T20:21:51.317863+0000","last_clean":"2024-04-26T20:21:51.317863+0000","last_became_active":"2024-04-26T20:21:48.872461+0000","last_became_peered":"2024-04-26T20:21:48.872461+0000","last_unstale":"2024-04-26T20:21:51.317863+0000","last_undegraded":"2024-04-26T20:21:51.317863+0000","last_fullsized":"2024-04-26T20:21:51.317863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888264+0000","last_change":"2024-04-26T20:21:49.885391+0000","last_active":"2024-04-26T20:21:50.888264+0000","last_peered":"2024-04-26T20:21:50.888264+0000","last_clean":"2024-04-26T20:21:50.888264+0000","last_became_active":"2024-04-26T20:21:49.884866+0000","last_became_peered":"2024-04-26T20:21:49.884866+0000","last_unstale":"2024-04-26T20:21:50.888264+0000","last_undegraded":"2024-04-26T20:21:50.888264+0000","last_fullsized":"2024-04-26T20:21:50.888264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888983+0000","last_change":"2024-04-26T20:21:48.873334+0000","last_active":"2024-04-26T20:21:50.888983+0000","last_peered":"2024-04-26T20:21:50.888983+0000","last_clean":"2024-04-26T20:21:50.888983+0000","last_became_active":"2024-04-26T20:21:48.872910+0000","last_became_peered":"2024-04-26T20:21:48.872910+0000","last_unstale":"2024-04-26T20:21:50.888983+0000","last_undegraded":"2024-04-26T20:21:50.888983+0000","last_fullsized":"2024-04-26T20:21:50.888983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:39.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317180+0000","last_change":"2024-04-26T20:21:49.889288+0000","last_active":"2024-04-26T20:21:51.317180+0000","last_peered":"2024-04-26T20:21:51.317180+0000","last_clean":"2024-04-26T20:21:51.317180+0000","last_became_active":"2024-04-26T20:21:49.889007+0000","last_became_peered":"2024-04-26T20:21:49.889007+0000","last_unstale":"2024-04-26T20:21:51.317180+0000","last_undegraded":"2024-04-26T20:21:51.317180+0000","last_fullsized":"2024-04-26T20:21:51.317180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:02.548978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891213+0000","last_change":"2024-04-26T20:21:48.875893+0000","last_active":"2024-04-26T20:21:50.891213+0000","last_peered":"2024-04-26T20:21:50.891213+0000","last_clean":"2024-04-26T20:21:50.891213+0000","last_became_active":"2024-04-26T20:21:48.875620+0000","last_became_peered":"2024-04-26T20:21:48.875620+0000","last_unstale":"2024-04-26T20:21:50.891213+0000","last_undegraded":"2024-04-26T20:21:50.891213+0000","last_fullsized":"2024-04-26T20:21:50.891213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317151+0000","last_change":"2024-04-26T20:21:49.895277+0000","last_active":"2024-04-26T20:21:51.317151+0000","last_peered":"2024-04-26T20:21:51.317151+0000","last_clean":"2024-04-26T20:21:51.317151+0000","last_became_active":"2024-04-26T20:21:49.894988+0000","last_became_peered":"2024-04-26T20:21:49.894988+0000","last_unstale":"2024-04-26T20:21:51.317151+0000","last_undegraded":"2024-04-26T20:21:51.317151+0000","last_fullsized":"2024-04-26T20:21:51.317151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:32.476892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888386+0000","last_change":"2024-04-26T20:21:48.881818+0000","last_active":"2024-04-26T20:21:50.888386+0000","last_peered":"2024-04-26T20:21:50.888386+0000","last_clean":"2024-04-26T20:21:50.888386+0000","last_became_active":"2024-04-26T20:21:48.881426+0000","last_became_peered":"2024-04-26T20:21:48.881426+0000","last_unstale":"2024-04-26T20:21:50.888386+0000","last_undegraded":"2024-04-26T20:21:50.888386+0000","last_fullsized":"2024-04-26T20:21:50.888386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:10.468097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888081+0000","last_change":"2024-04-26T20:21:49.891143+0000","last_active":"2024-04-26T20:21:50.888081+0000","last_peered":"2024-04-26T20:21:50.888081+0000","last_clean":"2024-04-26T20:21:50.888081+0000","last_became_active":"2024-04-26T20:21:49.890710+0000","last_became_peered":"2024-04-26T20:21:49.890710+0000","last_unstale":"2024-04-26T20:21:50.888081+0000","last_undegraded":"2024-04-26T20:21:50.888081+0000","last_fullsized":"2024-04-26T20:21:50.888081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318330+0000","last_change":"2024-04-26T20:21:48.872822+0000","last_active":"2024-04-26T20:21:51.318330+0000","last_peered":"2024-04-26T20:21:51.318330+0000","last_clean":"2024-04-26T20:21:51.318330+0000","last_became_active":"2024-04-26T20:21:48.872542+0000","last_became_peered":"2024-04-26T20:21:48.872542+0000","last_unstale":"2024-04-26T20:21:51.318330+0000","last_undegraded":"2024-04-26T20:21:51.318330+0000","last_fullsized":"2024-04-26T20:21:51.318330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318093+0000","last_change":"2024-04-26T20:21:49.894321+0000","last_active":"2024-04-26T20:21:51.318093+0000","last_peered":"2024-04-26T20:21:51.318093+0000","last_clean":"2024-04-26T20:21:51.318093+0000","last_became_active":"2024-04-26T20:21:49.893996+0000","last_became_peered":"2024-04-26T20:21:49.893996+0000","last_unstale":"2024-04-26T20:21:51.318093+0000","last_undegraded":"2024-04-26T20:21:51.318093+0000","last_fullsized":"2024-04-26T20:21:51.318093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:52.770156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891066+0000","last_change":"2024-04-26T20:21:48.879859+0000","last_active":"2024-04-26T20:21:50.891066+0000","last_peered":"2024-04-26T20:21:50.891066+0000","last_clean":"2024-04-26T20:21:50.891066+0000","last_became_active":"2024-04-26T20:21:48.879557+0000","last_became_peered":"2024-04-26T20:21:48.879557+0000","last_unstale":"2024-04-26T20:21:50.891066+0000","last_undegraded":"2024-04-26T20:21:50.891066+0000","last_fullsized":"2024-04-26T20:21:50.891066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891233+0000","last_change":"2024-04-26T20:21:49.902684+0000","last_active":"2024-04-26T20:21:50.891233+0000","last_peered":"2024-04-26T20:21:50.891233+0000","last_clean":"2024-04-26T20:21:50.891233+0000","last_became_active":"2024-04-26T20:21:49.902380+0000","last_became_peered":"2024-04-26T20:21:49.902380+0000","last_unstale":"2024-04-26T20:21:50.891233+0000","last_undegraded":"2024-04-26T20:21:50.891233+0000","last_fullsized":"2024-04-26T20:21:50.891233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890777+0000","last_change":"2024-04-26T20:21:48.877680+0000","last_active":"2024-04-26T20:21:50.890777+0000","last_peered":"2024-04-26T20:21:50.890777+0000","last_clean":"2024-04-26T20:21:50.890777+0000","last_became_active":"2024-04-26T20:21:48.877390+0000","last_became_peered":"2024-04-26T20:21:48.877390+0000","last_unstale":"2024-04-26T20:21:50.890777+0000","last_undegraded":"2024-04-26T20:21:50.890777+0000","last_fullsized":"2024-04-26T20:21:50.890777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316851+0000","last_change":"2024-04-26T20:21:49.896604+0000","last_active":"2024-04-26T20:21:51.316851+0000","last_peered":"2024-04-26T20:21:51.316851+0000","last_clean":"2024-04-26T20:21:51.316851+0000","last_became_active":"2024-04-26T20:21:49.894327+0000","last_became_peered":"2024-04-26T20:21:49.894327+0000","last_unstale":"2024-04-26T20:21:51.316851+0000","last_undegraded":"2024-04-26T20:21:51.316851+0000","last_fullsized":"2024-04-26T20:21:51.316851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:46.887612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318312+0000","last_change":"2024-04-26T20:21:48.880981+0000","last_active":"2024-04-26T20:21:51.318312+0000","last_peered":"2024-04-26T20:21:51.318312+0000","last_clean":"2024-04-26T20:21:51.318312+0000","last_became_active":"2024-04-26T20:21:48.880662+0000","last_became_peered":"2024-04-26T20:21:48.880662+0000","last_unstale":"2024-04-26T20:21:51.318312+0000","last_undegraded":"2024-04-26T20:21:51.318312+0000","last_fullsized":"2024-04-26T20:21:51.318312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:12.587571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317595+0000","last_change":"2024-04-26T20:21:49.894131+0000","last_active":"2024-04-26T20:21:51.317595+0000","last_peered":"2024-04-26T20:21:51.317595+0000","last_clean":"2024-04-26T20:21:51.317595+0000","last_became_active":"2024-04-26T20:21:49.893827+0000","last_became_peered":"2024-04-26T20:21:49.893827+0000","last_unstale":"2024-04-26T20:21:51.317595+0000","last_undegraded":"2024-04-26T20:21:51.317595+0000","last_fullsized":"2024-04-26T20:21:51.317595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:57.212796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890988+0000","last_change":"2024-04-26T20:21:48.875201+0000","last_active":"2024-04-26T20:21:50.890988+0000","last_peered":"2024-04-26T20:21:50.890988+0000","last_clean":"2024-04-26T20:21:50.890988+0000","last_became_active":"2024-04-26T20:21:48.874859+0000","last_became_peered":"2024-04-26T20:21:48.874859+0000","last_unstale":"2024-04-26T20:21:50.890988+0000","last_undegraded":"2024-04-26T20:21:50.890988+0000","last_fullsized":"2024-04-26T20:21:50.890988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317768+0000","last_change":"2024-04-26T20:21:49.893283+0000","last_active":"2024-04-26T20:21:51.317768+0000","last_peered":"2024-04-26T20:21:51.317768+0000","last_clean":"2024-04-26T20:21:51.317768+0000","last_became_active":"2024-04-26T20:21:49.893006+0000","last_became_peered":"2024-04-26T20:21:49.893006+0000","last_unstale":"2024-04-26T20:21:51.317768+0000","last_undegraded":"2024-04-26T20:21:51.317768+0000","last_fullsized":"2024-04-26T20:21:51.317768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:57.391564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888957+0000","last_change":"2024-04-26T20:21:48.880076+0000","last_active":"2024-04-26T20:21:50.888957+0000","last_peered":"2024-04-26T20:21:50.888957+0000","last_clean":"2024-04-26T20:21:50.888957+0000","last_became_active":"2024-04-26T20:21:48.879805+0000","last_became_peered":"2024-04-26T20:21:48.879805+0000","last_unstale":"2024-04-26T20:21:50.888957+0000","last_undegraded":"2024-04-26T20:21:50.888957+0000","last_fullsized":"2024-04-26T20:21:50.888957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:25.685618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318491+0000","last_change":"2024-04-26T20:21:49.895219+0000","last_active":"2024-04-26T20:21:51.318491+0000","last_peered":"2024-04-26T20:21:51.318491+0000","last_clean":"2024-04-26T20:21:51.318491+0000","last_became_active":"2024-04-26T20:21:49.894965+0000","last_became_peered":"2024-04-26T20:21:49.894965+0000","last_unstale":"2024-04-26T20:21:51.318491+0000","last_undegraded":"2024-04-26T20:21:51.318491+0000","last_fullsized":"2024-04-26T20:21:51.318491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:50.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890958+0000","last_change":"2024-04-26T20:21:48.878737+0000","last_active":"2024-04-26T20:21:50.890958+0000","last_peered":"2024-04-26T20:21:50.890958+0000","last_clean":"2024-04-26T20:21:50.890958+0000","last_became_active":"2024-04-26T20:21:48.878184+0000","last_became_peered":"2024-04-26T20:21:48.878184+0000","last_unstale":"2024-04-26T20:21:50.890958+0000","last_undegraded":"2024-04-26T20:21:50.890958+0000","last_fullsized":"2024-04-26T20:21:50.890958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:40:48.030416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317314+0000","last_change":"2024-04-26T20:21:49.897731+0000","last_active":"2024-04-26T20:21:51.317314+0000","last_peered":"2024-04-26T20:21:51.317314+0000","last_clean":"2024-04-26T20:21:51.317314+0000","last_became_active":"2024-04-26T20:21:49.897398+0000","last_became_peered":"2024-04-26T20:21:49.897398+0000","last_unstale":"2024-04-26T20:21:51.317314+0000","last_undegraded":"2024-04-26T20:21:51.317314+0000","last_fullsized":"2024-04-26T20:21:51.317314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.737636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317884+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:21:51.317884+0000","last_peered":"2024-04-26T20:21:51.317884+0000","last_clean":"2024-04-26T20:21:51.317884+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:21:51.317884+0000","last_undegraded":"2024-04-26T20:21:51.317884+0000","last_fullsized":"2024-04-26T20:21:51.317884+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317283+0000","last_change":"2024-04-26T20:21:48.873546+0000","last_active":"2024-04-26T20:21:51.317283+0000","last_peered":"2024-04-26T20:21:51.317283+0000","last_clean":"2024-04-26T20:21:51.317283+0000","last_became_active":"2024-04-26T20:21:48.873258+0000","last_became_peered":"2024-04-26T20:21:48.873258+0000","last_unstale":"2024-04-26T20:21:51.317283+0000","last_undegraded":"2024-04-26T20:21:51.317283+0000","last_fullsized":"2024-04-26T20:21:51.317283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316468+0000","last_change":"2024-04-26T20:21:49.892620+0000","last_active":"2024-04-26T20:21:51.316468+0000","last_peered":"2024-04-26T20:21:51.316468+0000","last_clean":"2024-04-26T20:21:51.316468+0000","last_became_active":"2024-04-26T20:21:49.891979+0000","last_became_peered":"2024-04-26T20:21:49.891979+0000","last_unstale":"2024-04-26T20:21:51.316468+0000","last_undegraded":"2024-04-26T20:21:51.316468+0000","last_fullsized":"2024-04-26T20:21:51.316468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317818+0000","last_change":"2024-04-26T20:21:48.883411+0000","last_active":"2024-04-26T20:21:51.317818+0000","last_peered":"2024-04-26T20:21:51.317818+0000","last_clean":"2024-04-26T20:21:51.317818+0000","last_became_active":"2024-04-26T20:21:48.883156+0000","last_became_peered":"2024-04-26T20:21:48.883156+0000","last_unstale":"2024-04-26T20:21:51.317818+0000","last_undegraded":"2024-04-26T20:21:51.317818+0000","last_fullsized":"2024-04-26T20:21:51.317818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:39.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888021+0000","last_change":"2024-04-26T20:21:49.889027+0000","last_active":"2024-04-26T20:21:50.888021+0000","last_peered":"2024-04-26T20:21:50.888021+0000","last_clean":"2024-04-26T20:21:50.888021+0000","last_became_active":"2024-04-26T20:21:49.888730+0000","last_became_peered":"2024-04-26T20:21:49.888730+0000","last_unstale":"2024-04-26T20:21:50.888021+0000","last_undegraded":"2024-04-26T20:21:50.888021+0000","last_fullsized":"2024-04-26T20:21:50.888021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890869+0000","last_change":"2024-04-26T20:21:48.875168+0000","last_active":"2024-04-26T20:21:50.890869+0000","last_peered":"2024-04-26T20:21:50.890869+0000","last_clean":"2024-04-26T20:21:50.890869+0000","last_became_active":"2024-04-26T20:21:48.874839+0000","last_became_peered":"2024-04-26T20:21:48.874839+0000","last_unstale":"2024-04-26T20:21:50.890869+0000","last_undegraded":"2024-04-26T20:21:50.890869+0000","last_fullsized":"2024-04-26T20:21:50.890869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888697+0000","last_change":"2024-04-26T20:21:49.895165+0000","last_active":"2024-04-26T20:21:50.888697+0000","last_peered":"2024-04-26T20:21:50.888697+0000","last_clean":"2024-04-26T20:21:50.888697+0000","last_became_active":"2024-04-26T20:21:49.894872+0000","last_became_peered":"2024-04-26T20:21:49.894872+0000","last_unstale":"2024-04-26T20:21:50.888697+0000","last_undegraded":"2024-04-26T20:21:50.888697+0000","last_fullsized":"2024-04-26T20:21:50.888697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318427+0000","last_change":"2024-04-26T20:21:48.874437+0000","last_active":"2024-04-26T20:21:51.318427+0000","last_peered":"2024-04-26T20:21:51.318427+0000","last_clean":"2024-04-26T20:21:51.318427+0000","last_became_active":"2024-04-26T20:21:48.874165+0000","last_became_peered":"2024-04-26T20:21:48.874165+0000","last_unstale":"2024-04-26T20:21:51.318427+0000","last_undegraded":"2024-04-26T20:21:51.318427+0000","last_fullsized":"2024-04-26T20:21:51.318427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317263+0000","last_change":"2024-04-26T20:21:49.894448+0000","last_active":"2024-04-26T20:21:51.317263+0000","last_peered":"2024-04-26T20:21:51.317263+0000","last_clean":"2024-04-26T20:21:51.317263+0000","last_became_active":"2024-04-26T20:21:49.894119+0000","last_became_peered":"2024-04-26T20:21:49.894119+0000","last_unstale":"2024-04-26T20:21:51.317263+0000","last_undegraded":"2024-04-26T20:21:51.317263+0000","last_fullsized":"2024-04-26T20:21:51.317263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:40.680121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888896+0000","last_change":"2024-04-26T20:21:48.874713+0000","last_active":"2024-04-26T20:21:50.888896+0000","last_peered":"2024-04-26T20:21:50.888896+0000","last_clean":"2024-04-26T20:21:50.888896+0000","last_became_active":"2024-04-26T20:21:48.874414+0000","last_became_peered":"2024-04-26T20:21:48.874414+0000","last_unstale":"2024-04-26T20:21:50.888896+0000","last_undegraded":"2024-04-26T20:21:50.888896+0000","last_fullsized":"2024-04-26T20:21:50.888896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317980+0000","last_change":"2024-04-26T20:21:49.895171+0000","last_active":"2024-04-26T20:21:51.317980+0000","last_peered":"2024-04-26T20:21:51.317980+0000","last_clean":"2024-04-26T20:21:51.317980+0000","last_became_active":"2024-04-26T20:21:49.894889+0000","last_became_peered":"2024-04-26T20:21:49.894889+0000","last_unstale":"2024-04-26T20:21:51.317980+0000","last_undegraded":"2024-04-26T20:21:51.317980+0000","last_fullsized":"2024-04-26T20:21:51.317980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:46.887612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317633+0000","last_change":"2024-04-26T20:21:48.882622+0000","last_active":"2024-04-26T20:21:51.317633+0000","last_peered":"2024-04-26T20:21:51.317633+0000","last_clean":"2024-04-26T20:21:51.317633+0000","last_became_active":"2024-04-26T20:21:48.882344+0000","last_became_peered":"2024-04-26T20:21:48.882344+0000","last_unstale":"2024-04-26T20:21:51.317633+0000","last_undegraded":"2024-04-26T20:21:51.317633+0000","last_fullsized":"2024-04-26T20:21:51.317633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:35.034364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888822+0000","last_change":"2024-04-26T20:21:49.894444+0000","last_active":"2024-04-26T20:21:50.888822+0000","last_peered":"2024-04-26T20:21:50.888822+0000","last_clean":"2024-04-26T20:21:50.888822+0000","last_became_active":"2024-04-26T20:21:49.894111+0000","last_became_peered":"2024-04-26T20:21:49.894111+0000","last_unstale":"2024-04-26T20:21:50.888822+0000","last_undegraded":"2024-04-26T20:21:50.888822+0000","last_fullsized":"2024-04-26T20:21:50.888822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890978+0000","last_change":"2024-04-26T20:21:48.872831+0000","last_active":"2024-04-26T20:21:50.890978+0000","last_peered":"2024-04-26T20:21:50.890978+0000","last_clean":"2024-04-26T20:21:50.890978+0000","last_became_active":"2024-04-26T20:21:48.872523+0000","last_became_peered":"2024-04-26T20:21:48.872523+0000","last_unstale":"2024-04-26T20:21:50.890978+0000","last_undegraded":"2024-04-26T20:21:50.890978+0000","last_fullsized":"2024-04-26T20:21:50.890978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316932+0000","last_change":"2024-04-26T20:21:49.888957+0000","last_active":"2024-04-26T20:21:51.316932+0000","last_peered":"2024-04-26T20:21:51.316932+0000","last_clean":"2024-04-26T20:21:51.316932+0000","last_became_active":"2024-04-26T20:21:49.888621+0000","last_became_peered":"2024-04-26T20:21:49.888621+0000","last_unstale":"2024-04-26T20:21:51.316932+0000","last_undegraded":"2024-04-26T20:21:51.316932+0000","last_fullsized":"2024-04-26T20:21:51.316932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:53.734129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891334+0000","last_change":"2024-04-26T20:21:48.878830+0000","last_active":"2024-04-26T20:21:50.891334+0000","last_peered":"2024-04-26T20:21:50.891334+0000","last_clean":"2024-04-26T20:21:50.891334+0000","last_became_active":"2024-04-26T20:21:48.878560+0000","last_became_peered":"2024-04-26T20:21:48.878560+0000","last_unstale":"2024-04-26T20:21:50.891334+0000","last_undegraded":"2024-04-26T20:21:50.891334+0000","last_fullsized":"2024-04-26T20:21:50.891334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888017+0000","last_change":"2024-04-26T20:21:49.893919+0000","last_active":"2024-04-26T20:21:50.888017+0000","last_peered":"2024-04-26T20:21:50.888017+0000","last_clean":"2024-04-26T20:21:50.888017+0000","last_became_active":"2024-04-26T20:21:49.893508+0000","last_became_peered":"2024-04-26T20:21:49.893508+0000","last_unstale":"2024-04-26T20:21:50.888017+0000","last_undegraded":"2024-04-26T20:21:50.888017+0000","last_fullsized":"2024-04-26T20:21:50.888017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:42.131574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317700+0000","last_change":"2024-04-26T20:21:48.883061+0000","last_active":"2024-04-26T20:21:51.317700+0000","last_peered":"2024-04-26T20:21:51.317700+0000","last_clean":"2024-04-26T20:21:51.317700+0000","last_became_active":"2024-04-26T20:21:48.882776+0000","last_became_peered":"2024-04-26T20:21:48.882776+0000","last_unstale":"2024-04-26T20:21:51.317700+0000","last_undegraded":"2024-04-26T20:21:51.317700+0000","last_fullsized":"2024-04-26T20:21:51.317700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316756+0000","last_change":"2024-04-26T20:21:49.892633+0000","last_active":"2024-04-26T20:21:51.316756+0000","last_peered":"2024-04-26T20:21:51.316756+0000","last_clean":"2024-04-26T20:21:51.316756+0000","last_became_active":"2024-04-26T20:21:49.892308+0000","last_became_peered":"2024-04-26T20:21:49.892308+0000","last_unstale":"2024-04-26T20:21:51.316756+0000","last_undegraded":"2024-04-26T20:21:51.316756+0000","last_fullsized":"2024-04-26T20:21:51.316756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:59.560594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888824+0000","last_change":"2024-04-26T20:21:48.878718+0000","last_active":"2024-04-26T20:21:50.888824+0000","last_peered":"2024-04-26T20:21:50.888824+0000","last_clean":"2024-04-26T20:21:50.888824+0000","last_became_active":"2024-04-26T20:21:48.878382+0000","last_became_peered":"2024-04-26T20:21:48.878382+0000","last_unstale":"2024-04-26T20:21:50.888824+0000","last_undegraded":"2024-04-26T20:21:50.888824+0000","last_fullsized":"2024-04-26T20:21:50.888824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316423+0000","last_change":"2024-04-26T20:21:49.893414+0000","last_active":"2024-04-26T20:21:51.316423+0000","last_peered":"2024-04-26T20:21:51.316423+0000","last_clean":"2024-04-26T20:21:51.316423+0000","last_became_active":"2024-04-26T20:21:49.893094+0000","last_became_peered":"2024-04-26T20:21:49.893094+0000","last_unstale":"2024-04-26T20:21:51.316423+0000","last_undegraded":"2024-04-26T20:21:51.316423+0000","last_fullsized":"2024-04-26T20:21:51.316423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888774+0000","last_change":"2024-04-26T20:21:48.874766+0000","last_active":"2024-04-26T20:21:50.888774+0000","last_peered":"2024-04-26T20:21:50.888774+0000","last_clean":"2024-04-26T20:21:50.888774+0000","last_became_active":"2024-04-26T20:21:48.874339+0000","last_became_peered":"2024-04-26T20:21:48.874339+0000","last_unstale":"2024-04-26T20:21:50.888774+0000","last_undegraded":"2024-04-26T20:21:50.888774+0000","last_fullsized":"2024-04-26T20:21:50.888774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316594+0000","last_change":"2024-04-26T20:21:49.893713+0000","last_active":"2024-04-26T20:21:51.316594+0000","last_peered":"2024-04-26T20:21:51.316594+0000","last_clean":"2024-04-26T20:21:51.316594+0000","last_became_active":"2024-04-26T20:21:49.891376+0000","last_became_peered":"2024-04-26T20:21:49.891376+0000","last_unstale":"2024-04-26T20:21:51.316594+0000","last_undegraded":"2024-04-26T20:21:51.316594+0000","last_fullsized":"2024-04-26T20:21:51.316594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318283+0000","last_change":"2024-04-26T20:21:48.882680+0000","last_active":"2024-04-26T20:21:51.318283+0000","last_peered":"2024-04-26T20:21:51.318283+0000","last_clean":"2024-04-26T20:21:51.318283+0000","last_became_active":"2024-04-26T20:21:48.882393+0000","last_became_peered":"2024-04-26T20:21:48.882393+0000","last_unstale":"2024-04-26T20:21:51.318283+0000","last_undegraded":"2024-04-26T20:21:51.318283+0000","last_fullsized":"2024-04-26T20:21:51.318283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:41.422336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891134+0000","last_change":"2024-04-26T20:21:49.892761+0000","last_active":"2024-04-26T20:21:50.891134+0000","last_peered":"2024-04-26T20:21:50.891134+0000","last_clean":"2024-04-26T20:21:50.891134+0000","last_became_active":"2024-04-26T20:21:49.892403+0000","last_became_peered":"2024-04-26T20:21:49.892403+0000","last_unstale":"2024-04-26T20:21:50.891134+0000","last_undegraded":"2024-04-26T20:21:50.891134+0000","last_fullsized":"2024-04-26T20:21:50.891134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317815+0000","last_change":"2024-04-26T20:21:49.890488+0000","last_active":"2024-04-26T20:21:51.317815+0000","last_peered":"2024-04-26T20:21:51.317815+0000","last_clean":"2024-04-26T20:21:51.317815+0000","last_became_active":"2024-04-26T20:21:49.889950+0000","last_became_peered":"2024-04-26T20:21:49.889950+0000","last_unstale":"2024-04-26T20:21:51.317815+0000","last_undegraded":"2024-04-26T20:21:51.317815+0000","last_fullsized":"2024-04-26T20:21:51.317815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317232+0000","last_change":"2024-04-26T20:21:48.884202+0000","last_active":"2024-04-26T20:21:51.317232+0000","last_peered":"2024-04-26T20:21:51.317232+0000","last_clean":"2024-04-26T20:21:51.317232+0000","last_became_active":"2024-04-26T20:21:48.883945+0000","last_became_peered":"2024-04-26T20:21:48.883945+0000","last_unstale":"2024-04-26T20:21:51.317232+0000","last_undegraded":"2024-04-26T20:21:51.317232+0000","last_fullsized":"2024-04-26T20:21:51.317232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:25.685618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318377+0000","last_change":"2024-04-26T20:21:49.894650+0000","last_active":"2024-04-26T20:21:51.318377+0000","last_peered":"2024-04-26T20:21:51.318377+0000","last_clean":"2024-04-26T20:21:51.318377+0000","last_became_active":"2024-04-26T20:21:49.889860+0000","last_became_peered":"2024-04-26T20:21:49.889860+0000","last_unstale":"2024-04-26T20:21:51.318377+0000","last_undegraded":"2024-04-26T20:21:51.318377+0000","last_fullsized":"2024-04-26T20:21:51.318377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:13.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317454+0000","last_change":"2024-04-26T20:21:48.881647+0000","last_active":"2024-04-26T20:21:51.317454+0000","last_peered":"2024-04-26T20:21:51.317454+0000","last_clean":"2024-04-26T20:21:51.317454+0000","last_became_active":"2024-04-26T20:21:48.881349+0000","last_became_peered":"2024-04-26T20:21:48.881349+0000","last_unstale":"2024-04-26T20:21:51.317454+0000","last_undegraded":"2024-04-26T20:21:51.317454+0000","last_fullsized":"2024-04-26T20:21:51.317454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:24.800032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888111+0000","last_change":"2024-04-26T20:21:49.884044+0000","last_active":"2024-04-26T20:21:50.888111+0000","last_peered":"2024-04-26T20:21:50.888111+0000","last_clean":"2024-04-26T20:21:50.888111+0000","last_became_active":"2024-04-26T20:21:49.883750+0000","last_became_peered":"2024-04-26T20:21:49.883750+0000","last_unstale":"2024-04-26T20:21:50.888111+0000","last_undegraded":"2024-04-26T20:21:50.888111+0000","last_fullsized":"2024-04-26T20:21:50.888111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316858+0000","last_change":"2024-04-26T20:21:48.875662+0000","last_active":"2024-04-26T20:21:51.316858+0000","last_peered":"2024-04-26T20:21:51.316858+0000","last_clean":"2024-04-26T20:21:51.316858+0000","last_became_active":"2024-04-26T20:21:48.875024+0000","last_became_peered":"2024-04-26T20:21:48.875024+0000","last_unstale":"2024-04-26T20:21:51.316858+0000","last_undegraded":"2024-04-26T20:21:51.316858+0000","last_fullsized":"2024-04-26T20:21:51.316858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:40:48.030416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316941+0000","last_change":"2024-04-26T20:21:49.894476+0000","last_active":"2024-04-26T20:21:51.316941+0000","last_peered":"2024-04-26T20:21:51.316941+0000","last_clean":"2024-04-26T20:21:51.316941+0000","last_became_active":"2024-04-26T20:21:49.894186+0000","last_became_peered":"2024-04-26T20:21:49.894186+0000","last_unstale":"2024-04-26T20:21:51.316941+0000","last_undegraded":"2024-04-26T20:21:51.316941+0000","last_fullsized":"2024-04-26T20:21:51.316941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:27.648550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888239+0000","last_change":"2024-04-26T20:21:48.880562+0000","last_active":"2024-04-26T20:21:50.888239+0000","last_peered":"2024-04-26T20:21:50.888239+0000","last_clean":"2024-04-26T20:21:50.888239+0000","last_became_active":"2024-04-26T20:21:48.880249+0000","last_became_peered":"2024-04-26T20:21:48.880249+0000","last_unstale":"2024-04-26T20:21:50.888239+0000","last_undegraded":"2024-04-26T20:21:50.888239+0000","last_fullsized":"2024-04-26T20:21:50.888239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:21:54.254 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:21:54.254 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T20:21:54.254 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T20:21:54.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T20:21:54.255 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T20:21:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.389+0000 7fc1c6615640 1 Processor -- start 2024-04-26T20:21:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.389+0000 7fc1c6615640 1 -- start start 2024-04-26T20:21:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.389+0000 7fc1c6615640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c015cf70 0x7fc1c006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.389+0000 7fc1c6615640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc1c006fbf0 0x7fc1c006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.389+0000 7fc1c6615640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c0070210 con 0x7fc1c0163b70 2024-04-26T20:21:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.390+0000 7fc1c6615640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c0163090 con 0x7fc1c006fbf0 2024-04-26T20:21:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.390+0000 7fc1c6615640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c0163370 con 0x7fc1c015cf70 2024-04-26T20:21:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.390+0000 7fc1bffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c015cf70 0x7fc1c006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.390+0000 7fc1c4bb2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc1c006fbf0 0x7fc1c006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.390+0000 7fc1bffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c015cf70 0x7fc1c006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:36640/0 (socket says 172.21.15.37:36640) 2024-04-26T20:21:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.390+0000 7fc1bffff640 1 -- 172.21.15.37:0/1700776723 learned_addr learned my addr 172.21.15.37:0/1700776723 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:54.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.390+0000 7fc1bffff640 1 -- 172.21.15.37:0/1700776723 >> v2:172.21.15.134:3300/0 conn(0x7fc1c006fbf0 msgr2=0x7fc1c006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:54.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.390+0000 7fc1bffff640 1 --2- 172.21.15.37:0/1700776723 >> v2:172.21.15.134:3300/0 conn(0x7fc1c006fbf0 0x7fc1c006c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.390+0000 7fc1bffff640 1 -- 172.21.15.37:0/1700776723 >> v1:172.21.15.134:6789/0 conn(0x7fc1c0163b70 legacy=0x7fc1c006f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.390+0000 7fc1bffff640 1 -- 172.21.15.37:0/1700776723 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c0163650 con 0x7fc1c015cf70 2024-04-26T20:21:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.390+0000 7fc1c4bb2640 1 --2- 172.21.15.37:0/1700776723 >> v2:172.21.15.134:3300/0 conn(0x7fc1c006fbf0 0x7fc1c006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.391+0000 7fc1bffff640 1 --2- 172.21.15.37:0/1700776723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c015cf70 0x7fc1c006c240 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009870 tx=0x7fc1b002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75ab5040ad9a85a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.391+0000 7fc1beffd640 1 -- 172.21.15.37:0/1700776723 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0037070 con 0x7fc1c015cf70 2024-04-26T20:21:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.391+0000 7fc1beffd640 1 -- 172.21.15.37:0/1700776723 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b0032440 con 0x7fc1c015cf70 2024-04-26T20:21:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.391+0000 7fc1beffd640 1 -- 172.21.15.37:0/1700776723 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0032810 con 0x7fc1c015cf70 2024-04-26T20:21:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.392+0000 7fc1c6615640 1 -- 172.21.15.37:0/1700776723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c015cf70 msgr2=0x7fc1c006c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.392+0000 7fc1c6615640 1 --2- 172.21.15.37:0/1700776723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c015cf70 0x7fc1c006c240 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009870 tx=0x7fc1b002a270 comp rx=0 tx=0).stop 2024-04-26T20:21:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.392+0000 7fc1c6615640 1 -- 172.21.15.37:0/1700776723 shutdown_connections 2024-04-26T20:21:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.392+0000 7fc1c6615640 1 --2- 172.21.15.37:0/1700776723 >> v2:172.21.15.134:3300/0 conn(0x7fc1c006fbf0 0x7fc1c006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.392+0000 7fc1c6615640 1 --2- 172.21.15.37:0/1700776723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c015cf70 0x7fc1c006c240 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.392+0000 7fc1c6615640 1 -- 172.21.15.37:0/1700776723 >> 172.21.15.37:0/1700776723 conn(0x7fc1c00b3b00 msgr2=0x7fc1c00b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:54.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.392+0000 7fc1c6615640 1 -- 172.21.15.37:0/1700776723 shutdown_connections 2024-04-26T20:21:54.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.392+0000 7fc1c6615640 1 -- 172.21.15.37:0/1700776723 wait complete. 2024-04-26T20:21:54.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.393+0000 7fc1c6615640 1 Processor -- start 2024-04-26T20:21:54.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.393+0000 7fc1c6615640 1 -- start start 2024-04-26T20:21:54.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.394+0000 7fc1c6615640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c006fbf0 0x7fc1c01f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.394+0000 7fc1c6615640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c015cf70 0x7fc1c0161c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.394+0000 7fc1c6615640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c0163b70 0x7fc1c00af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.394+0000 7fc1c6615640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c015fbc0 con 0x7fc1c015cf70 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.394+0000 7fc1c6615640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c015fa40 con 0x7fc1c0163b70 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.394+0000 7fc1c6615640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c015fd40 con 0x7fc1c006fbf0 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.394+0000 7fc1bf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c015cf70 0x7fc1c0161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.394+0000 7fc1bf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c015cf70 0x7fc1c0161c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59550/0 (socket says 172.21.15.37:59550) 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.394+0000 7fc1bf7fe640 1 -- 172.21.15.37:0/543350170 learned_addr learned my addr 172.21.15.37:0/543350170 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.394+0000 7fc1bffff640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c006fbf0 0x7fc1c01f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.394+0000 7fc1c4bb2640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c0163b70 0x7fc1c00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.395+0000 7fc1bf7fe640 1 -- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c006fbf0 msgr2=0x7fc1c01f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.395+0000 7fc1bf7fe640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c006fbf0 0x7fc1c01f7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.395+0000 7fc1bf7fe640 1 -- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c0163b70 msgr2=0x7fc1c00af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.395+0000 7fc1bf7fe640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c0163b70 0x7fc1c00af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.395+0000 7fc1bf7fe640 1 -- 172.21.15.37:0/543350170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c01f84f0 con 0x7fc1c015cf70 2024-04-26T20:21:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.395+0000 7fc1bf7fe640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c015cf70 0x7fc1c0161c70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400c3f0 tx=0x7fc1b400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.395+0000 7fc1bcff9640 1 -- 172.21.15.37:0/543350170 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40052d0 con 0x7fc1c015cf70 2024-04-26T20:21:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.395+0000 7fc1bcff9640 1 -- 172.21.15.37:0/543350170 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b4005490 con 0x7fc1c015cf70 2024-04-26T20:21:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.395+0000 7fc1c6615640 1 -- 172.21.15.37:0/543350170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c01e91d0 con 0x7fc1c015cf70 2024-04-26T20:21:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.395+0000 7fc1bcff9640 1 -- 172.21.15.37:0/543350170 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4007260 con 0x7fc1c015cf70 2024-04-26T20:21:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.395+0000 7fc1c6615640 1 -- 172.21.15.37:0/543350170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c01e9790 con 0x7fc1c015cf70 2024-04-26T20:21:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.396+0000 7fc1c6615640 1 -- 172.21.15.37:0/543350170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c00693e0 con 0x7fc1c015cf70 2024-04-26T20:21:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.396+0000 7fc1bcff9640 1 -- 172.21.15.37:0/543350170 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1b4007400 con 0x7fc1c015cf70 2024-04-26T20:21:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.397+0000 7fc1bcff9640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1ac06c920 0x7fc1ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.397+0000 7fc1bcff9640 1 -- 172.21.15.37:0/543350170 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc1b408aff0 con 0x7fc1c015cf70 2024-04-26T20:21:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.397+0000 7fc1bffff640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1ac06c920 0x7fc1ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.398+0000 7fc1bffff640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1ac06c920 0x7fc1ac06ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0034000 tx=0x7fc1b002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:54.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.401+0000 7fc1bcff9640 1 -- 172.21.15.37:0/543350170 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1b4059420 con 0x7fc1c015cf70 2024-04-26T20:21:54.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.636+0000 7fc1c6615640 1 -- 172.21.15.37:0/543350170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc1c00b34a0 con 0x7fc1c015cf70 2024-04-26T20:21:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.639+0000 7fc1bcff9640 1 -- 172.21.15.37:0/543350170 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174142 (secure 0 0 0) 0x7fc1b4058d20 con 0x7fc1c015cf70 2024-04-26T20:21:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 -- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1ac06c920 msgr2=0x7fc1ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1ac06c920 0x7fc1ac06ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0034000 tx=0x7fc1b002e000 comp rx=0 tx=0).stop 2024-04-26T20:21:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 -- 172.21.15.37:0/543350170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c015cf70 msgr2=0x7fc1c0161c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c015cf70 0x7fc1c0161c70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400c3f0 tx=0x7fc1b400c8b0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 -- 172.21.15.37:0/543350170 shutdown_connections 2024-04-26T20:21:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c0163b70 0x7fc1c00af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c015cf70 0x7fc1c0161c70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1ac06c920 0x7fc1ac06ede0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 --2- 172.21.15.37:0/543350170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c006fbf0 0x7fc1c01f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 -- 172.21.15.37:0/543350170 >> 172.21.15.37:0/543350170 conn(0x7fc1c00b3b00 msgr2=0x7fc1c00b5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:54.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 -- 172.21.15.37:0/543350170 shutdown_connections 2024-04-26T20:21:54.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.642+0000 7fc1c6615640 1 -- 172.21.15.37:0/543350170 wait complete. 2024-04-26T20:21:54.653 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2709823969},{"type":"v1","addr":"172.21.15.134:6833","nonce":2709823969}]},"active_addr":"172.21.15.134:6833/2709823969","active_change":"2024-04-26T20:21:29.323840+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":2860386177}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":66230988}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":1727150548}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":1002023369}]}]} 2024-04-26T20:21:54.657 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T20:21:54.657 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T20:21:54.657 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:21:54.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.756+0000 7f0f43746640 1 Processor -- start 2024-04-26T20:21:54.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.756+0000 7f0f43746640 1 -- start start 2024-04-26T20:21:54.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f43746640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3c15f380 0x7f0f3c0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f43746640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0f3c0b9a90 0x7f0f3c0d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f43746640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f3c0d3de0 con 0x7f0f3c15f760 2024-04-26T20:21:54.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f43746640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f0f3c0d43a0 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f43746640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3c0d40c0 con 0x7f0f3c15f380 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f41ce3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0f3c0b9a90 0x7f0f3c0d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f41ce3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0f3c0b9a90 0x7f0f3c0d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59566/0 (socket says 172.21.15.37:59566) 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f40ce1640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0f3c15f760 0x7f0f3c0cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52746/0 (socket says 172.21.15.37:52746) 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f41ce3640 1 -- 172.21.15.37:0/3018932223 learned_addr learned my addr 172.21.15.37:0/3018932223 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f414e2640 1 --2- 172.21.15.37:0/3018932223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3c15f380 0x7f0f3c0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f41ce3640 1 -- 172.21.15.37:0/3018932223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3c15f380 msgr2=0x7f0f3c0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f41ce3640 1 --2- 172.21.15.37:0/3018932223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3c15f380 0x7f0f3c0d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f41ce3640 1 -- 172.21.15.37:0/3018932223 >> v1:172.21.15.37:6789/0 conn(0x7f0f3c15f760 legacy=0x7f0f3c0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f41ce3640 1 -- 172.21.15.37:0/3018932223 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c0b8280 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.757+0000 7f0f33fff640 1 -- 172.21.15.37:0/3018932223 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2912623795 0 0) 0x7f0f3c0d3de0 con 0x7f0f3c15f760 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.758+0000 7f0f41ce3640 1 --2- 172.21.15.37:0/3018932223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f3c0b9a90 0x7f0f3c0d2110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009e50 tx=0x7f0f3402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d60b95dfcd31c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.758+0000 7f0f33fff640 1 -- 172.21.15.37:0/3018932223 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34036070 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.758+0000 7f0f33fff640 1 -- 172.21.15.37:0/3018932223 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f34031600 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.758+0000 7f0f33fff640 1 -- 172.21.15.37:0/3018932223 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f340319d0 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.759+0000 7f0f43746640 1 -- 172.21.15.37:0/3018932223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f3c0b9a90 msgr2=0x7f0f3c0d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.759+0000 7f0f43746640 1 --2- 172.21.15.37:0/3018932223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f3c0b9a90 0x7f0f3c0d2110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009e50 tx=0x7f0f3402a7a0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.759+0000 7f0f43746640 1 -- 172.21.15.37:0/3018932223 shutdown_connections 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.759+0000 7f0f43746640 1 --2- 172.21.15.37:0/3018932223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f3c0b9a90 0x7f0f3c0d2110 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.759+0000 7f0f43746640 1 --2- 172.21.15.37:0/3018932223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3c15f380 0x7f0f3c0d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.759+0000 7f0f43746640 1 -- 172.21.15.37:0/3018932223 >> 172.21.15.37:0/3018932223 conn(0x7f0f3c09e560 msgr2=0x7f0f3c0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.759+0000 7f0f43746640 1 -- 172.21.15.37:0/3018932223 shutdown_connections 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.759+0000 7f0f43746640 1 -- 172.21.15.37:0/3018932223 wait complete. 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.760+0000 7f0f43746640 1 Processor -- start 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.760+0000 7f0f43746640 1 -- start start 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.760+0000 7f0f43746640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f3c0b9a90 0x7f0f3c154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f43746640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3c15f380 0x7f0f3c161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f43746640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3c15f760 0x7f0f3c1624a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f43746640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c0d1280 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f43746640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c0d1100 con 0x7f0f3c15f760 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f43746640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3c0d5b50 con 0x7f0f3c15f380 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f414e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f3c0b9a90 0x7f0f3c154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f414e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f3c0b9a90 0x7f0f3c154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59576/0 (socket says 172.21.15.37:59576) 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f414e2640 1 -- 172.21.15.37:0/1526437970 learned_addr learned my addr 172.21.15.37:0/1526437970 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f40ce1640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3c15f380 0x7f0f3c161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f41ce3640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3c15f760 0x7f0f3c1624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f414e2640 1 -- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3c15f380 msgr2=0x7f0f3c161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f414e2640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3c15f380 0x7f0f3c161f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f414e2640 1 -- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3c15f760 msgr2=0x7f0f3c1624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f414e2640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3c15f760 0x7f0f3c1624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f414e2640 1 -- 172.21.15.37:0/1526437970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c0c96d0 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f414e2640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f3c0b9a90 0x7f0f3c154b40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800d800 tx=0x7f0f3800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f31ffb640 1 -- 172.21.15.37:0/1526437970 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800b6c0 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f31ffb640 1 -- 172.21.15.37:0/1526437970 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f38019070 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f31ffb640 1 -- 172.21.15.37:0/1526437970 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800ba80 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.761+0000 7f0f43746640 1 -- 172.21.15.37:0/1526437970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c0c99b0 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.762+0000 7f0f43746640 1 -- 172.21.15.37:0/1526437970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c160a00 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.763+0000 7f0f43746640 1 -- 172.21.15.37:0/1526437970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f3c0d0c30 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.763+0000 7f0f31ffb640 1 -- 172.21.15.37:0/1526437970 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f3800bc20 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.763+0000 7f0f31ffb640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0f2006c8f0 0x7f0f2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.763+0000 7f0f31ffb640 1 -- 172.21.15.37:0/1526437970 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0f3808c040 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.763+0000 7f0f40ce1640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0f2006c8f0 0x7f0f2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.764+0000 7f0f40ce1640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0f2006c8f0 0x7f0f2006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c0cf270 tx=0x7f0f2c007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:54.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.768+0000 7f0f31ffb640 1 -- 172.21.15.37:0/1526437970 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f3c0d0c30 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.953+0000 7f0f43746640 1 -- 172.21.15.37:0/1526437970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f3c06a360 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.954+0000 7f0f31ffb640 1 -- 172.21.15.37:0/1526437970 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f0f3801c360 con 0x7f0f3c0b9a90 2024-04-26T20:21:54.956 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:54.956 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:21:54.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 -- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0f2006c8f0 msgr2=0x7f0f2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0f2006c8f0 0x7f0f2006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c0cf270 tx=0x7f0f2c007c00 comp rx=0 tx=0).stop 2024-04-26T20:21:54.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 -- 172.21.15.37:0/1526437970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f3c0b9a90 msgr2=0x7f0f3c154b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f3c0b9a90 0x7f0f3c154b40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800d800 tx=0x7f0f3800dcc0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 -- 172.21.15.37:0/1526437970 shutdown_connections 2024-04-26T20:21:54.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3c15f760 0x7f0f3c1624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0f2006c8f0 0x7f0f2006edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3c15f380 0x7f0f3c161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 --2- 172.21.15.37:0/1526437970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f3c0b9a90 0x7f0f3c154b40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 -- 172.21.15.37:0/1526437970 >> 172.21.15.37:0/1526437970 conn(0x7f0f3c09e560 msgr2=0x7f0f3c0cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:54.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 -- 172.21.15.37:0/1526437970 shutdown_connections 2024-04-26T20:21:54.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:54.957+0000 7f0f43746640 1 -- 172.21.15.37:0/1526437970 wait complete. 2024-04-26T20:21:54.969 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T20:21:54.969 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:21:55.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.068+0000 7fc58dbc6640 1 Processor -- start 2024-04-26T20:21:55.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.068+0000 7fc58dbc6640 1 -- start start 2024-04-26T20:21:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.068+0000 7fc58dbc6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc58815f080 0x7fc58806d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.068+0000 7fc58dbc6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc58815f460 0x7fc5880cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.068+0000 7fc58dbc6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc58806c3e0 con 0x7fc5880ca8e0 2024-04-26T20:21:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.068+0000 7fc58dbc6640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc58806d9c0 con 0x7fc58815f460 2024-04-26T20:21:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.068+0000 7fc58dbc6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc58806c6c0 con 0x7fc58815f080 2024-04-26T20:21:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.069+0000 7fc586ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc58815f460 0x7fc5880cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.069+0000 7fc586ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc58815f460 0x7fc5880cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59580/0 (socket says 172.21.15.37:59580) 2024-04-26T20:21:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.069+0000 7fc586ffd640 1 -- 172.21.15.37:0/547109599 learned_addr learned my addr 172.21.15.37:0/547109599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.069+0000 7fc5877fe640 1 --2- 172.21.15.37:0/547109599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc58815f080 0x7fc58806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.069+0000 7fc586ffd640 1 -- 172.21.15.37:0/547109599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc58815f080 msgr2=0x7fc58806d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.069+0000 7fc586ffd640 1 --2- 172.21.15.37:0/547109599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc58815f080 0x7fc58806d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.069+0000 7fc586ffd640 1 -- 172.21.15.37:0/547109599 >> v1:172.21.15.134:6789/0 conn(0x7fc5880ca8e0 legacy=0x7fc5880cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.069+0000 7fc586ffd640 1 -- 172.21.15.37:0/547109599 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58806c9a0 con 0x7fc58815f460 2024-04-26T20:21:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc586ffd640 1 --2- 172.21.15.37:0/547109599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc58815f460 0x7fc5880cd7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009870 tx=0x7fc57c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716cc59d7292749e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc5867fc640 1 -- 172.21.15.37:0/547109599 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c037070 con 0x7fc58815f460 2024-04-26T20:21:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc5867fc640 1 -- 172.21.15.37:0/547109599 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c032440 con 0x7fc58815f460 2024-04-26T20:21:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc5867fc640 1 -- 172.21.15.37:0/547109599 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c032810 con 0x7fc58815f460 2024-04-26T20:21:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc58dbc6640 1 -- 172.21.15.37:0/547109599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc58815f460 msgr2=0x7fc5880cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc58dbc6640 1 --2- 172.21.15.37:0/547109599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc58815f460 0x7fc5880cd7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009870 tx=0x7fc57c02a210 comp rx=0 tx=0).stop 2024-04-26T20:21:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc58dbc6640 1 -- 172.21.15.37:0/547109599 shutdown_connections 2024-04-26T20:21:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc58dbc6640 1 --2- 172.21.15.37:0/547109599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc58815f460 0x7fc5880cd7c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc58dbc6640 1 --2- 172.21.15.37:0/547109599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc58815f080 0x7fc58806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc58dbc6640 1 -- 172.21.15.37:0/547109599 >> 172.21.15.37:0/547109599 conn(0x7fc5880ce050 msgr2=0x7fc5880ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc58dbc6640 1 -- 172.21.15.37:0/547109599 shutdown_connections 2024-04-26T20:21:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.070+0000 7fc58dbc6640 1 -- 172.21.15.37:0/547109599 wait complete. 2024-04-26T20:21:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.071+0000 7fc58dbc6640 1 Processor -- start 2024-04-26T20:21:55.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.072+0000 7fc58dbc6640 1 -- start start 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.072+0000 7fc58dbc6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5880ca8e0 0x7fc5881410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.072+0000 7fc58dbc6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58815f080 0x7fc5881438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.072+0000 7fc58dbc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc58815f460 0x7fc588160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.072+0000 7fc58dbc6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5880cdd00 con 0x7fc58815f460 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.072+0000 7fc58dbc6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc58806cf60 con 0x7fc58815f080 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.072+0000 7fc58dbc6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc5880cde80 con 0x7fc5880ca8e0 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc587fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc58815f460 0x7fc588160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc587fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc58815f460 0x7fc588160950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59594/0 (socket says 172.21.15.37:59594) 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc587fff640 1 -- 172.21.15.37:0/536690360 learned_addr learned my addr 172.21.15.37:0/536690360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc5877fe640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5880ca8e0 0x7fc5881410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc586ffd640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58815f080 0x7fc5881438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc587fff640 1 -- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5880ca8e0 msgr2=0x7fc5881410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc587fff640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5880ca8e0 0x7fc5881410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc587fff640 1 -- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58815f080 msgr2=0x7fc5881438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc587fff640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58815f080 0x7fc5881438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc587fff640 1 -- 172.21.15.37:0/536690360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5881442b0 con 0x7fc58815f460 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc586ffd640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58815f080 0x7fc5881438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc587fff640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc58815f460 0x7fc588160950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc58000c780 tx=0x7fc58000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc56ffff640 1 -- 172.21.15.37:0/536690360 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc580004b60 con 0x7fc58815f460 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc56ffff640 1 -- 172.21.15.37:0/536690360 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc580004d00 con 0x7fc58815f460 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc58dbc6640 1 -- 172.21.15.37:0/536690360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc588144590 con 0x7fc58815f460 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc56ffff640 1 -- 172.21.15.37:0/536690360 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58000aa90 con 0x7fc58815f460 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.073+0000 7fc58dbc6640 1 -- 172.21.15.37:0/536690360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58815d7c0 con 0x7fc58815f460 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.074+0000 7fc58dbc6640 1 -- 172.21.15.37:0/536690360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc58806a360 con 0x7fc58815f460 2024-04-26T20:21:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.075+0000 7fc56ffff640 1 -- 172.21.15.37:0/536690360 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc58000ac30 con 0x7fc58815f460 2024-04-26T20:21:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.075+0000 7fc56ffff640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc57406c8f0 0x7fc57406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.075+0000 7fc56ffff640 1 -- 172.21.15.37:0/536690360 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc58008bdf0 con 0x7fc58815f460 2024-04-26T20:21:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.075+0000 7fc5877fe640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc57406c8f0 0x7fc57406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.076+0000 7fc5877fe640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc57406c8f0 0x7fc57406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc57800a970 tx=0x7fc578008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.080+0000 7fc56ffff640 1 -- 172.21.15.37:0/536690360 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc580020070 con 0x7fc58815f460 2024-04-26T20:21:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.265+0000 7fc58dbc6640 1 -- 172.21.15.37:0/536690360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5880b7840 con 0x7fc58815f460 2024-04-26T20:21:55.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.266+0000 7fc56ffff640 1 -- 172.21.15.37:0/536690360 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7fc58005a220 con 0x7fc58815f460 2024-04-26T20:21:55.267 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:55.268 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:21:55.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 -- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc57406c8f0 msgr2=0x7fc57406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc57406c8f0 0x7fc57406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc57800a970 tx=0x7fc578008040 comp rx=0 tx=0).stop 2024-04-26T20:21:55.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 -- 172.21.15.37:0/536690360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc58815f460 msgr2=0x7fc588160950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc58815f460 0x7fc588160950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc58000c780 tx=0x7fc58000cc40 comp rx=0 tx=0).stop 2024-04-26T20:21:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 -- 172.21.15.37:0/536690360 shutdown_connections 2024-04-26T20:21:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc58815f460 0x7fc588160950 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58815f080 0x7fc5881438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc57406c8f0 0x7fc57406edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 --2- 172.21.15.37:0/536690360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5880ca8e0 0x7fc5881410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 -- 172.21.15.37:0/536690360 >> 172.21.15.37:0/536690360 conn(0x7fc5880ce050 msgr2=0x7fc58815b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 -- 172.21.15.37:0/536690360 shutdown_connections 2024-04-26T20:21:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.268+0000 7fc58dbc6640 1 -- 172.21.15.37:0/536690360 wait complete. 2024-04-26T20:21:55.280 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T20:21:55.281 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T20:21:55.281 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T20:21:55.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T20:21:55.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T20:21:55.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T20:21:55.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T20:21:55.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T20:21:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.456+0000 7f32f0796640 1 Processor -- start 2024-04-26T20:21:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.457+0000 7f424d164640 1 Processor -- start 2024-04-26T20:21:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.457+0000 7f424d164640 1 -- start start 2024-04-26T20:21:55.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.456+0000 7ff93d54a640 1 Processor -- start 2024-04-26T20:21:55.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.456+0000 7ff93d54a640 1 -- start start 2024-04-26T20:21:55.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.457+0000 7f3f8605b640 1 Processor -- start 2024-04-26T20:21:55.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7fa4f3fff640 1 Processor -- start 2024-04-26T20:21:55.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7fa4f3fff640 1 -- start start 2024-04-26T20:21:55.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7f424d164640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f42400ead00 0x7f424004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.457+0000 7f32f0796640 1 -- start start 2024-04-26T20:21:55.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7ff93d54a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff9380cfc10 0x7ff93806dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7ff93d54a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff93806fa70 con 0x7ff93806f690 2024-04-26T20:21:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7ff93d54a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9380d0070 con 0x7ff938167250 2024-04-26T20:21:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7ff93d54a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff938163c10 con 0x7ff9380cfc10 2024-04-26T20:21:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7f424d164640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f424004ffd0 0x7f4240037bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7f424d164640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4240053720 con 0x7f4240038110 2024-04-26T20:21:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7f424d164640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4240054330 con 0x7f42400ead00 2024-04-26T20:21:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7f424d164640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4240053a00 con 0x7f424004ffd0 2024-04-26T20:21:55.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7f62e76c8640 1 Processor -- start 2024-04-26T20:21:55.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7f9ca40a9640 1 Processor -- start 2024-04-26T20:21:55.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.458+0000 7f9ca40a9640 1 -- start start 2024-04-26T20:21:55.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7ff937fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff93806f690 0x7ff938164400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52752/0 (socket says 172.21.15.37:52752) 2024-04-26T20:21:55.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7ff937fff640 1 -- 172.21.15.37:0/2538468142 learned_addr learned my addr 172.21.15.37:0/2538468142 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7ff9377fe640 1 --2- 172.21.15.37:0/2538468142 >> v2:172.21.15.134:3300/0 conn(0x7ff9380cfc10 0x7ff93806dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f4247fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f42400ead00 0x7f424004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f424c963640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f4240038110 0x7f4240038580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:59794/0 (socket says 172.21.15.37:59794) 2024-04-26T20:21:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f424c963640 1 -- 172.21.15.37:0/2207572932 learned_addr learned my addr 172.21.15.37:0/2207572932 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f42477fe640 1 --2- 172.21.15.37:0/2207572932 >> v2:172.21.15.134:3300/0 conn(0x7f424004ffd0 0x7f4240037bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f62e76c8640 1 -- start start 2024-04-26T20:21:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.457+0000 7f3f8605b640 1 -- start start 2024-04-26T20:21:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f3f8605b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 0x7f3f80167300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f3f8605b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3f80167840 0x7f3f800c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f3f8605b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f80168130 con 0x7f3f80163690 2024-04-26T20:21:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f3f8605b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3f800cedd0 con 0x7f3f80167840 2024-04-26T20:21:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f3f8605b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3f800c5cd0 con 0x7f3f800ce860 2024-04-26T20:21:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 728733114 0 0) 0x7f4240053720 con 0x7f4240038110 2024-04-26T20:21:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4238003520 con 0x7f4240038110 2024-04-26T20:21:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4282306626 0 0) 0x7ff9380d0070 con 0x7ff938167250 2024-04-26T20:21:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff920003520 con 0x7ff938167250 2024-04-26T20:21:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f32f0796640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00e0120 0x7f32e00eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f32f0796640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e00512f0 con 0x7f32e00dfd40 2024-04-26T20:21:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f32f0796640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e00549d0 con 0x7f32e0050500 2024-04-26T20:21:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f32f0796640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f32e0054cb0 con 0x7f32e00e0120 2024-04-26T20:21:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f32eed33640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00e0120 0x7f32e00eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f32eed33640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00e0120 0x7f32e00eaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:36704/0 (socket says 172.21.15.37:36704) 2024-04-26T20:21:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f32eed33640 1 -- 172.21.15.37:0/1699347923 learned_addr learned my addr 172.21.15.37:0/1699347923 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1361019539 0 0) 0x7ff920003520 con 0x7ff938167250 2024-04-26T20:21:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 >> v2:172.21.15.134:3300/0 conn(0x7ff9380cfc10 msgr2=0x7ff93806dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 --2- 172.21.15.37:0/2538468142 >> v2:172.21.15.134:3300/0 conn(0x7ff9380cfc10 0x7ff93806dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 >> v1:172.21.15.37:6789/0 conn(0x7ff93806f690 legacy=0x7ff938164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93806e3d0 con 0x7ff938167250 2024-04-26T20:21:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7ff9300019f0 con 0x7ff938167250 2024-04-26T20:21:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 >> v1:172.21.15.134:6789/0 conn(0x7ff938167250 legacy=0x7ff93806c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4261310126 0 0) 0x7f4238003520 con 0x7f4240038110 2024-04-26T20:21:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 >> v2:172.21.15.134:3300/0 conn(0x7f424004ffd0 msgr2=0x7f4240037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f4246ffd640 1 --2- 172.21.15.37:0/2207572932 >> v2:172.21.15.134:3300/0 conn(0x7f424004ffd0 0x7f4240037bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 >> v2:172.21.15.37:3300/0 conn(0x7f42400ead00 msgr2=0x7f424004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f4246ffd640 1 --2- 172.21.15.37:0/2207572932 >> v2:172.21.15.37:3300/0 conn(0x7f42400ead00 0x7f424004fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4240053ce0 con 0x7f4240038110 2024-04-26T20:21:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f4248050f10 con 0x7f4240038110 2024-04-26T20:21:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 >> v1:172.21.15.134:6789/0 conn(0x7f4240038110 legacy=0x7f4240038580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f4246ffd640 1 --2- 172.21.15.37:0/2207572932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42380039b0 0x7f4238003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f4246ffd640 1 --2- 172.21.15.37:0/2207572932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4238004350 0x7f4238004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f4246ffd640 1 --2- 172.21.15.37:0/2207572932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4238004e90 0x7f423800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4240054330 con 0x7f4238004350 2024-04-26T20:21:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4240053a00 con 0x7f4238004e90 2024-04-26T20:21:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f423800bf00 con 0x7f42380039b0 2024-04-26T20:21:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f42477fe640 1 --2- 172.21.15.37:0/2207572932 >> v2:172.21.15.134:3300/0 conn(0x7f424004ffd0 0x7f4240037bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f424c963640 1 --2- 172.21.15.37:0/2207572932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4238004e90 0x7f423800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f9ca40a9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 0x7f9c94038170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f9ca40a9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9c9404f0b0 0x7f9c9404fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f9ca40a9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c94005f60 con 0x7f9c9400c0d0 2024-04-26T20:21:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f9ca40a9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f9c940df4d0 con 0x7f9c9404f0b0 2024-04-26T20:21:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f9ca40a9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9c940e38e0 con 0x7f9c940546c0 2024-04-26T20:21:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 --2- 172.21.15.37:0/2538468142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff920003a00 0x7ff920003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 --2- 172.21.15.37:0/2538468142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff920004800 0x7ff920004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 --2- 172.21.15.37:0/2538468142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9200051c0 0x7ff92000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f3f7effd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 0x7f3f80167300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f3f7effd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 0x7f3f80167300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:36700/0 (socket says 172.21.15.37:36700) 2024-04-26T20:21:55.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f3f7effd640 1 -- 172.21.15.37:0/214184130 learned_addr learned my addr 172.21.15.37:0/214184130 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4f40b4f70 0x7fa4f40d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7fa4f3fff640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f40d0e10 con 0x7fa4f41625a0 2024-04-26T20:21:55.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7fa4f3fff640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f40d2dc0 con 0x7fa4f4162980 2024-04-26T20:21:55.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7fa4f3fff640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f40d0be0 con 0x7fa4f40b4f70 2024-04-26T20:21:55.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.459+0000 7f3f7ffff640 1 --2- 172.21.15.37:0/214184130 >> v2:172.21.15.134:3300/0 conn(0x7f3f80167840 0x7f3f800c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f3f7effd640 1 -- 172.21.15.37:0/214184130 >> v2:172.21.15.134:3300/0 conn(0x7f3f80167840 msgr2=0x7f3f800c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f3f7effd640 1 --2- 172.21.15.37:0/214184130 >> v2:172.21.15.134:3300/0 conn(0x7f3f80167840 0x7f3f800c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f3f7effd640 1 -- 172.21.15.37:0/214184130 >> v1:172.21.15.134:6789/0 conn(0x7f3f80163690 legacy=0x7f3f800d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f3f7effd640 1 -- 172.21.15.37:0/214184130 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f800c5fb0 con 0x7f3f800ce860 2024-04-26T20:21:55.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff938163c10 con 0x7ff920004800 2024-04-26T20:21:55.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff92000c0c0 con 0x7ff9200051c0 2024-04-26T20:21:55.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff92000c2c0 con 0x7ff920003a00 2024-04-26T20:21:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f4247fff640 1 --2- 172.21.15.37:0/2207572932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42380039b0 0x7f4238003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7fd3a4447640 1 Processor -- start 2024-04-26T20:21:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f9ca1644640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f9c9400c0d0 0x7f9c940dedc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:59836/0 (socket says 172.21.15.37:59836) 2024-04-26T20:21:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7ff937fff640 1 --2- 172.21.15.37:0/2538468142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff920003a00 0x7ff920003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f62e76c8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f62e0163690 0x7f62e00d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f62e76c8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f62e00ce860 0x7f62e0167300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f62e76c8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e0168130 con 0x7f62e0167840 2024-04-26T20:21:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f62e76c8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f62e0167cb0 con 0x7f62e0163690 2024-04-26T20:21:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f62e76c8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f62e00d0a80 con 0x7f62e00ce860 2024-04-26T20:21:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f9ca1644640 1 -- 172.21.15.37:0/3768245698 learned_addr learned my addr 172.21.15.37:0/3768245698 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7fd3a4447640 1 -- start start 2024-04-26T20:21:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7fa4f37fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4f40b4f70 0x7fa4f40d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7f9ca1e45640 1 --2- 172.21.15.37:0/3768245698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 0x7f9c94038170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f62e4c63640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f62e00ce860 0x7f62e0167300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f62e4c63640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f62e00ce860 0x7f62e0167300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:43204/0 (socket says 172.21.15.37:43204) 2024-04-26T20:21:55.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f62e4c63640 1 -- 172.21.15.37:0/1806952624 learned_addr learned my addr 172.21.15.37:0/1806952624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7fa4f37fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4f40b4f70 0x7fa4f40d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:36726/0 (socket says 172.21.15.37:36726) 2024-04-26T20:21:55.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.460+0000 7fa4f37fe640 1 -- 172.21.15.37:0/1798037228 learned_addr learned my addr 172.21.15.37:0/1798037228 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7ff9377fe640 1 --2- 172.21.15.37:0/2538468142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff920004800 0x7ff920004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7ff93cd49640 1 --2- 172.21.15.37:0/2538468142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9200051c0 0x7ff92000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7ff937fff640 1 -- 172.21.15.37:0/2538468142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9200051c0 msgr2=0x7ff92000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7ff937fff640 1 --2- 172.21.15.37:0/2538468142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9200051c0 0x7ff92000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f9ca0e43640 1 -- 172.21.15.37:0/3768245698 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2086812161 0 0) 0x7f9c94005f60 con 0x7f9c9400c0d0 2024-04-26T20:21:55.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f9ca0e43640 1 -- 172.21.15.37:0/3768245698 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c88003520 con 0x7f9c9400c0d0 2024-04-26T20:21:55.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7ff937fff640 1 -- 172.21.15.37:0/2538468142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff920004800 msgr2=0x7ff920004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7ff937fff640 1 --2- 172.21.15.37:0/2538468142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff920004800 0x7ff920004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7ff937fff640 1 -- 172.21.15.37:0/2538468142 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff920003520 con 0x7ff920003a00 2024-04-26T20:21:55.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f62e5464640 1 --2- 172.21.15.37:0/1806952624 >> v2:172.21.15.37:3300/0 conn(0x7f62e0163690 0x7f62e00d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f9ca2646640 1 --2- 172.21.15.37:0/3768245698 >> v2:172.21.15.37:3300/0 conn(0x7f9c9404f0b0 0x7f9c9404fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f42477fe640 1 --2- 172.21.15.37:0/2207572932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4238004350 0x7f4238004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f32eed33640 1 -- 172.21.15.37:0/1699347923 >> v1:172.21.15.134:6789/0 conn(0x7f32e0050500 legacy=0x7f32e0007bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f32eed33640 1 -- 172.21.15.37:0/1699347923 >> v1:172.21.15.37:6789/0 conn(0x7f32e00dfd40 legacy=0x7f32e0050be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f32eed33640 1 -- 172.21.15.37:0/1699347923 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e0054f90 con 0x7f32e00e0120 2024-04-26T20:21:55.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7ff93cd49640 1 -- 172.21.15.37:0/2538468142 reap_dead start 2024-04-26T20:21:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f4247fff640 1 -- 172.21.15.37:0/2207572932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4238004e90 msgr2=0x7f423800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f4247fff640 1 --2- 172.21.15.37:0/2207572932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4238004e90 0x7f423800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f4247fff640 1 -- 172.21.15.37:0/2207572932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4238004350 msgr2=0x7f4238004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f4247fff640 1 --2- 172.21.15.37:0/2207572932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4238004350 0x7f4238004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f4247fff640 1 -- 172.21.15.37:0/2207572932 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4238003520 con 0x7f42380039b0 2024-04-26T20:21:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f424c963640 1 -- 172.21.15.37:0/2207572932 reap_dead start 2024-04-26T20:21:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7fa4f37fe640 1 -- 172.21.15.37:0/1798037228 >> v1:172.21.15.134:6789/0 conn(0x7fa4f4162980 legacy=0x7fa4f40b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7fa4f37fe640 1 -- 172.21.15.37:0/1798037228 >> v1:172.21.15.37:6789/0 conn(0x7fa4f41625a0 legacy=0x7fa4f40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7fa4f37fe640 1 -- 172.21.15.37:0/1798037228 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f40d30a0 con 0x7fa4f40b4f70 2024-04-26T20:21:55.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7fd3a4447640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39404f860 0x7fd39404fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f62e4c63640 1 -- 172.21.15.37:0/1806952624 >> v2:172.21.15.37:3300/0 conn(0x7f62e0163690 msgr2=0x7f62e00d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f62e4c63640 1 --2- 172.21.15.37:0/1806952624 >> v2:172.21.15.37:3300/0 conn(0x7f62e0163690 0x7f62e00d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f62e4c63640 1 -- 172.21.15.37:0/1806952624 >> v1:172.21.15.134:6789/0 conn(0x7f62e0167840 legacy=0x7f62e00add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.461+0000 7f62e4c63640 1 -- 172.21.15.37:0/1806952624 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e00cedd0 con 0x7f62e00ce860 2024-04-26T20:21:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7f3f7effd640 1 --2- 172.21.15.37:0/214184130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 0x7f3f80167300 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009870 tx=0x7f3f7402a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fe39b2f103647b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7fd3a4447640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd394037ba0 0x7fd394038030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7fd3a4447640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd394053690 con 0x7fd3940e32c0 2024-04-26T20:21:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7fa4f37fe640 1 --2- 172.21.15.37:0/1798037228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4f40b4f70 0x7fa4f40d06a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009eb0 tx=0x7fa4ec02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a656f5609621eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7fd3a4447640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fd3940e36a0 con 0x7fd394037ba0 2024-04-26T20:21:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7ff937fff640 1 --2- 172.21.15.37:0/2538468142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff920003a00 0x7ff920003e20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff928001810 tx=0x7ff920005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dc02165caa3366e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7f62e4c63640 1 --2- 172.21.15.37:0/1806952624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00ce860 0x7f62e0167300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62e006d220 tx=0x7f62d802a760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0613a4bfd1ebc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7f9ca1e45640 1 -- 172.21.15.37:0/3768245698 >> v2:172.21.15.37:3300/0 conn(0x7f9c9404f0b0 msgr2=0x7f9c9404fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7f9ca1e45640 1 --2- 172.21.15.37:0/3768245698 >> v2:172.21.15.37:3300/0 conn(0x7f9c9404f0b0 0x7f9c9404fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7f9ca1e45640 1 -- 172.21.15.37:0/3768245698 >> v1:172.21.15.134:6789/0 conn(0x7f9c9400c0d0 legacy=0x7f9c940dedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7f9ca1e45640 1 -- 172.21.15.37:0/3768245698 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c94006200 con 0x7f9c940546c0 2024-04-26T20:21:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7f3f7e7fc640 1 -- 172.21.15.37:0/214184130 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74037070 con 0x7f3f800ce860 2024-04-26T20:21:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928001460 con 0x7ff920003a00 2024-04-26T20:21:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92802e070 con 0x7ff920003a00 2024-04-26T20:21:55.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7ff936ffd640 1 -- 172.21.15.37:0/2538468142 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92802ad90 con 0x7ff920003a00 2024-04-26T20:21:55.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f62d7fff640 1 -- 172.21.15.37:0/1806952624 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8036070 con 0x7f62e00ce860 2024-04-26T20:21:55.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f62d7fff640 1 -- 172.21.15.37:0/1806952624 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d8031600 con 0x7f62e00ce860 2024-04-26T20:21:55.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f62d7fff640 1 -- 172.21.15.37:0/1806952624 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8031a10 con 0x7f62e00ce860 2024-04-26T20:21:55.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f3f7e7fc640 1 -- 172.21.15.37:0/214184130 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f74032440 con 0x7f3f800ce860 2024-04-26T20:21:55.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7fa4f1ffb640 1 -- 172.21.15.37:0/1798037228 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec036070 con 0x7fa4f40b4f70 2024-04-26T20:21:55.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7fa4f1ffb640 1 -- 172.21.15.37:0/1798037228 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec031600 con 0x7fa4f40b4f70 2024-04-26T20:21:55.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7fa4f1ffb640 1 -- 172.21.15.37:0/1798037228 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0319d0 con 0x7fa4f40b4f70 2024-04-26T20:21:55.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7fd3a4447640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd394053970 con 0x7fd39404f860 2024-04-26T20:21:55.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7fd3a21e3640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd3940e32c0 0x7fd394052c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52772/0 (socket says 172.21.15.37:52772) 2024-04-26T20:21:55.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.462+0000 7fd3a21e3640 1 -- 172.21.15.37:0/3387507344 learned_addr learned my addr 172.21.15.37:0/3387507344 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:55.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7fd3a29e4640 1 --2- 172.21.15.37:0/3387507344 >> v2:172.21.15.37:3300/0 conn(0x7fd394037ba0 0x7fd394038030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7fd3a11e1640 1 -- 172.21.15.37:0/3387507344 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 527106115 0 0) 0x7fd394053690 con 0x7fd3940e32c0 2024-04-26T20:21:55.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7fd3a11e1640 1 -- 172.21.15.37:0/3387507344 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd388003520 con 0x7fd3940e32c0 2024-04-26T20:21:55.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7fd3a19e2640 1 --2- 172.21.15.37:0/3387507344 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39404f860 0x7fd39404fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f9ca1e45640 1 --2- 172.21.15.37:0/3768245698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 0x7f9c94038170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9c980097a0 tx=0x7f9c9802a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61218980bd102c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f32eed33640 1 --2- 172.21.15.37:0/1699347923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00e0120 0x7f32e00eaa70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32d800bb40 tx=0x7f32d802c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e49a2f18faaf24da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f32ed530640 1 -- 172.21.15.37:0/1699347923 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8036020 con 0x7f32e00e0120 2024-04-26T20:21:55.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f32ed530640 1 -- 172.21.15.37:0/1699347923 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d8032810 con 0x7f32e00e0120 2024-04-26T20:21:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f4247fff640 1 --2- 172.21.15.37:0/2207572932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42380039b0 0x7f4238003dd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f423c009ac0 tx=0x7f4238005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=576467b9a41d974 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f32f0796640 1 -- 172.21.15.37:0/1699347923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00e0120 msgr2=0x7f32e00eaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f32f0796640 1 --2- 172.21.15.37:0/1699347923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00e0120 0x7f32e00eaa70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32d800bb40 tx=0x7f32d802c950 comp rx=0 tx=0).stop 2024-04-26T20:21:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7ff93d54a640 1 -- 172.21.15.37:0/2538468142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff920003a00 msgr2=0x7ff920003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7ff93d54a640 1 --2- 172.21.15.37:0/2538468142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff920003a00 0x7ff920003e20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff928001810 tx=0x7ff920005830 comp rx=0 tx=0).stop 2024-04-26T20:21:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7ff93d54a640 1 -- 172.21.15.37:0/2538468142 shutdown_connections 2024-04-26T20:21:55.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7ff93d54a640 1 --2- 172.21.15.37:0/2538468142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff920003a00 0x7ff920003e20 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff928001810 tx=0x7ff920005830 comp rx=0 tx=0).stop 2024-04-26T20:21:55.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7ff93d54a640 1 -- 172.21.15.37:0/2538468142 >> 172.21.15.37:0/2538468142 conn(0x7ff9380b2890 msgr2=0x7ff9380b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7ff93d54a640 1 -- 172.21.15.37:0/2538468142 shutdown_connections 2024-04-26T20:21:55.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7ff93d54a640 1 -- 172.21.15.37:0/2538468142 wait complete. 2024-04-26T20:21:55.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f62e76c8640 1 -- 172.21.15.37:0/1806952624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00ce860 msgr2=0x7f62e0167300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f62e76c8640 1 --2- 172.21.15.37:0/1806952624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00ce860 0x7f62e0167300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62e006d220 tx=0x7f62d802a760 comp rx=0 tx=0).stop 2024-04-26T20:21:55.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f62e76c8640 1 -- 172.21.15.37:0/1806952624 shutdown_connections 2024-04-26T20:21:55.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f62e76c8640 1 --2- 172.21.15.37:0/1806952624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00ce860 0x7f62e0167300 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62e006d220 tx=0x7f62d802a760 comp rx=0 tx=0).stop 2024-04-26T20:21:55.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f62e76c8640 1 --2- 172.21.15.37:0/1806952624 >> v2:172.21.15.37:3300/0 conn(0x7f62e0163690 0x7f62e00d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f62e76c8640 1 -- 172.21.15.37:0/1806952624 >> 172.21.15.37:0/1806952624 conn(0x7f62e00d0c90 msgr2=0x7f62e00d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f62e76c8640 1 -- 172.21.15.37:0/1806952624 shutdown_connections 2024-04-26T20:21:55.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c036070 con 0x7f42380039b0 2024-04-26T20:21:55.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f423c031400 con 0x7f42380039b0 2024-04-26T20:21:55.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f4246ffd640 1 -- 172.21.15.37:0/2207572932 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c009690 con 0x7f42380039b0 2024-04-26T20:21:55.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f424d164640 1 -- 172.21.15.37:0/2207572932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42380039b0 msgr2=0x7f4238003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f424d164640 1 --2- 172.21.15.37:0/2207572932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42380039b0 0x7f4238003dd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f423c009ac0 tx=0x7f4238005830 comp rx=0 tx=0).stop 2024-04-26T20:21:55.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f9ca0e43640 1 -- 172.21.15.37:0/3768245698 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98037070 con 0x7f9c940546c0 2024-04-26T20:21:55.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f9ca0e43640 1 -- 172.21.15.37:0/3768245698 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c98032440 con 0x7f9c940546c0 2024-04-26T20:21:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7f9ca0e43640 1 -- 172.21.15.37:0/3768245698 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98032810 con 0x7f9c940546c0 2024-04-26T20:21:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f9ca40a9640 1 -- 172.21.15.37:0/3768245698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 msgr2=0x7f9c94038170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f9ca40a9640 1 --2- 172.21.15.37:0/3768245698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 0x7f9c94038170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9c980097a0 tx=0x7f9c9802a160 comp rx=0 tx=0).stop 2024-04-26T20:21:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f3f8605b640 1 -- 172.21.15.37:0/214184130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 msgr2=0x7f3f80167300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f3f8605b640 1 --2- 172.21.15.37:0/214184130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 0x7f3f80167300 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009870 tx=0x7f3f7402a200 comp rx=0 tx=0).stop 2024-04-26T20:21:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f3f8605b640 1 -- 172.21.15.37:0/214184130 shutdown_connections 2024-04-26T20:21:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f3f8605b640 1 --2- 172.21.15.37:0/214184130 >> v2:172.21.15.134:3300/0 conn(0x7f3f80167840 0x7f3f800c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f3f8605b640 1 --2- 172.21.15.37:0/214184130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 0x7f3f80167300 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f3f8605b640 1 -- 172.21.15.37:0/214184130 >> 172.21.15.37:0/214184130 conn(0x7f3f800d0c90 msgr2=0x7f3f800d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7fa4f3fff640 1 -- 172.21.15.37:0/1798037228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4f40b4f70 msgr2=0x7fa4f40d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7fa4f3fff640 1 --2- 172.21.15.37:0/1798037228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4f40b4f70 0x7fa4f40d06a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009eb0 tx=0x7fa4ec02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7fa4f3fff640 1 -- 172.21.15.37:0/1798037228 shutdown_connections 2024-04-26T20:21:55.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7fa4f3fff640 1 --2- 172.21.15.37:0/1798037228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4f40b4f70 0x7fa4f40d06a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7fa4f3fff640 1 -- 172.21.15.37:0/1798037228 >> 172.21.15.37:0/1798037228 conn(0x7fa4f40d4fa0 msgr2=0x7fa4f40d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7fa4f3fff640 1 -- 172.21.15.37:0/1798037228 shutdown_connections 2024-04-26T20:21:55.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7fd3a11e1640 1 -- 172.21.15.37:0/3387507344 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2031005850 0 0) 0x7fd388003520 con 0x7fd3940e32c0 2024-04-26T20:21:55.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7fd3a29e4640 1 -- 172.21.15.37:0/3387507344 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39404f860 msgr2=0x7fd39404fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7fd3a29e4640 1 --2- 172.21.15.37:0/3387507344 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39404f860 0x7fd39404fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7fd3a29e4640 1 -- 172.21.15.37:0/3387507344 >> v1:172.21.15.37:6789/0 conn(0x7fd3940e32c0 legacy=0x7fd394052c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.463+0000 7fd3a29e4640 1 -- 172.21.15.37:0/3387507344 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd394053c50 con 0x7fd394037ba0 2024-04-26T20:21:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f32f0796640 1 -- 172.21.15.37:0/1699347923 shutdown_connections 2024-04-26T20:21:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f32f0796640 1 --2- 172.21.15.37:0/1699347923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00e0120 0x7f32e00eaa70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f32f0796640 1 -- 172.21.15.37:0/1699347923 >> 172.21.15.37:0/1699347923 conn(0x7f32e0055bc0 msgr2=0x7f32e0055fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f3f8605b640 1 -- 172.21.15.37:0/214184130 shutdown_connections 2024-04-26T20:21:55.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f3f8605b640 1 -- 172.21.15.37:0/214184130 wait complete. 2024-04-26T20:21:55.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7f9ca40a9640 1 -- 172.21.15.37:0/3768245698 shutdown_connections 2024-04-26T20:21:55.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7f9ca40a9640 1 --2- 172.21.15.37:0/3768245698 >> v2:172.21.15.37:3300/0 conn(0x7f9c9404f0b0 0x7f9c9404fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7f9ca40a9640 1 --2- 172.21.15.37:0/3768245698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 0x7f9c94038170 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7f9ca40a9640 1 -- 172.21.15.37:0/3768245698 >> 172.21.15.37:0/3768245698 conn(0x7f9c94009fe0 msgr2=0x7f9c9400a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7fd3a29e4640 1 --2- 172.21.15.37:0/3387507344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394037ba0 0x7fd394038030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0097a0 tx=0x7fd38c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d036493df749742 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7fa4f3fff640 1 -- 172.21.15.37:0/1798037228 wait complete. 2024-04-26T20:21:55.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.464+0000 7f62e76c8640 1 -- 172.21.15.37:0/1806952624 wait complete. 2024-04-26T20:21:55.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7f424d164640 1 -- 172.21.15.37:0/2207572932 shutdown_connections 2024-04-26T20:21:55.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7f424d164640 1 --2- 172.21.15.37:0/2207572932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42380039b0 0x7f4238003dd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7f424d164640 1 -- 172.21.15.37:0/2207572932 >> 172.21.15.37:0/2207572932 conn(0x7f4240009fb0 msgr2=0x7f424000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7fd3a11e1640 1 -- 172.21.15.37:0/3387507344 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c037070 con 0x7fd394037ba0 2024-04-26T20:21:55.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7f9ca40a9640 1 -- 172.21.15.37:0/3768245698 shutdown_connections 2024-04-26T20:21:55.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7fd3a11e1640 1 -- 172.21.15.37:0/3387507344 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38c02ab90 con 0x7fd394037ba0 2024-04-26T20:21:55.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7fd3a11e1640 1 -- 172.21.15.37:0/3387507344 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398001670 con 0x7fd394037ba0 2024-04-26T20:21:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.465+0000 7f9ca40a9640 1 -- 172.21.15.37:0/3768245698 wait complete. 2024-04-26T20:21:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7ff93d54a640 1 Processor -- start 2024-04-26T20:21:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7f62e76c8640 1 Processor -- start 2024-04-26T20:21:55.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7f424d164640 1 -- 172.21.15.37:0/2207572932 shutdown_connections 2024-04-26T20:21:55.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7ff93d54a640 1 -- start start 2024-04-26T20:21:55.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7f3f8605b640 1 Processor -- start 2024-04-26T20:21:55.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7fd3a4447640 1 -- 172.21.15.37:0/3387507344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394037ba0 msgr2=0x7fd394038030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7fd3a4447640 1 --2- 172.21.15.37:0/3387507344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394037ba0 0x7fd394038030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0097a0 tx=0x7fd38c02a100 comp rx=0 tx=0).stop 2024-04-26T20:21:55.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7f424d164640 1 -- 172.21.15.37:0/2207572932 wait complete. 2024-04-26T20:21:55.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7fd3a4447640 1 -- 172.21.15.37:0/3387507344 shutdown_connections 2024-04-26T20:21:55.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7fd3a4447640 1 --2- 172.21.15.37:0/3387507344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394037ba0 0x7fd394038030 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7fd3a4447640 1 --2- 172.21.15.37:0/3387507344 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39404f860 0x7fd39404fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7fd3a4447640 1 -- 172.21.15.37:0/3387507344 >> 172.21.15.37:0/3387507344 conn(0x7fd394009fa0 msgr2=0x7fd39400a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f9ca40a9640 1 Processor -- start 2024-04-26T20:21:55.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f32f0796640 1 -- 172.21.15.37:0/1699347923 shutdown_connections 2024-04-26T20:21:55.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7fd3a4447640 1 -- 172.21.15.37:0/3387507344 shutdown_connections 2024-04-26T20:21:55.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f32f0796640 1 -- 172.21.15.37:0/1699347923 wait complete. 2024-04-26T20:21:55.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7f62e76c8640 1 -- start start 2024-04-26T20:21:55.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f62e76c8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62e0163690 0x7f62e0155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f62e76c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e0167840 0x7f62e009db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f62e76c8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e009e0d0 0x7f62e009ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f62e76c8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f62e00c6290 con 0x7f62e0167840 2024-04-26T20:21:55.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f62e76c8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62e00c5fb0 con 0x7f62e009e0d0 2024-04-26T20:21:55.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f62e76c8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f62e0163a70 con 0x7f62e0163690 2024-04-26T20:21:55.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7f3f8605b640 1 -- start start 2024-04-26T20:21:55.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f62e5c65640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e009e0d0 0x7f62e009ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f62e5c65640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e009e0d0 0x7f62e009ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:43206/0 (socket says 172.21.15.37:43206) 2024-04-26T20:21:55.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f62e5c65640 1 -- 172.21.15.37:0/2394549067 learned_addr learned my addr 172.21.15.37:0/2394549067 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f3f8605b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 0x7f3f800c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7ff93d54a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff938156010 0x7ff938159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7ff93d54a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9381fa150 0x7ff93815c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7ff93d54a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff93815c9b0 0x7ff9381fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7ff937fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff938156010 0x7ff938159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7ff937fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff938156010 0x7ff938159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59692/0 (socket says 172.21.15.37:59692) 2024-04-26T20:21:55.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7ff93d54a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff938163ee0 con 0x7ff938156010 2024-04-26T20:21:55.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7ff93d54a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff93806e990 con 0x7ff9381fa150 2024-04-26T20:21:55.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f62e5c65640 1 -- 172.21.15.37:0/2394549067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62e0163690 msgr2=0x7f62e0155a60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:21:55.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f62e5c65640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62e0163690 0x7f62e0155a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f62e5c65640 1 -- 172.21.15.37:0/2394549067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e0167840 msgr2=0x7f62e009db90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T20:21:55.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7f9ca40a9640 1 -- start start 2024-04-26T20:21:55.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f9ca40a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c9400c0d0 0x7f9c9404bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f9ca40a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c9404f0b0 0x7f9c940ddc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f9ca40a9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 0x7f9c940de170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f9ca40a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94050640 con 0x7f9c9404f0b0 2024-04-26T20:21:55.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f3f8605b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f80163690 0x7f3f8009caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f3f8605b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f80167840 0x7f3f801474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f3f8605b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80163bf0 con 0x7f3f80167840 2024-04-26T20:21:55.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f3f8605b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80163a70 con 0x7f3f80163690 2024-04-26T20:21:55.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f3f8605b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3f800c6cf0 con 0x7f3f800ce860 2024-04-26T20:21:55.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f3f7effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f80163690 0x7f3f8009caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f3f7effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f80163690 0x7f3f8009caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:43226/0 (socket says 172.21.15.37:43226) 2024-04-26T20:21:55.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f3f7effd640 1 -- 172.21.15.37:0/1674363528 learned_addr learned my addr 172.21.15.37:0/1674363528 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7fa4f3fff640 1 Processor -- start 2024-04-26T20:21:55.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.466+0000 7fa4f3fff640 1 -- start start 2024-04-26T20:21:55.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f40b4f70 0x7fa4f40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4f41625a0 0x7fa4f40cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4f4162980 0x7fa4f406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.467+0000 7fd3a4447640 1 -- 172.21.15.37:0/3387507344 wait complete. 2024-04-26T20:21:55.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7ff93cd49640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff93815c9b0 0x7ff9381fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7ff93cd49640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff93815c9b0 0x7ff9381fcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:36748/0 (socket says 172.21.15.37:36748) 2024-04-26T20:21:55.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7ff93cd49640 1 -- 172.21.15.37:0/884124353 learned_addr learned my addr 172.21.15.37:0/884124353 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f424d164640 1 Processor -- start 2024-04-26T20:21:55.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f9ca40a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c940e3bc0 con 0x7f9c9400c0d0 2024-04-26T20:21:55.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f9ca40a9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9c940507c0 con 0x7f9c940546c0 2024-04-26T20:21:55.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7fa4f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f40b4f70 0x7fa4f40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7fa4f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f40b4f70 0x7fa4f40cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59662/0 (socket says 172.21.15.37:59662) 2024-04-26T20:21:55.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7fa4f2ffd640 1 -- 172.21.15.37:0/3157646589 learned_addr learned my addr 172.21.15.37:0/3157646589 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f40d3410 con 0x7fa4f40b4f70 2024-04-26T20:21:55.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7fa4f37fe640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4f4162980 0x7fa4f406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f40d2b70 con 0x7fa4f4162980 2024-04-26T20:21:55.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f416a140 con 0x7fa4f41625a0 2024-04-26T20:21:55.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7ff93d54a640 1 -- 172.21.15.37:0/884124353 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9381676e0 con 0x7ff93815c9b0 2024-04-26T20:21:55.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f62e4c63640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e0167840 0x7f62e009db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f9ca2646640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 0x7f9c940de170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f9ca2646640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 0x7f9c940de170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:36764/0 (socket says 172.21.15.37:36764) 2024-04-26T20:21:55.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f9ca2646640 1 -- 172.21.15.37:0/1031912952 learned_addr learned my addr 172.21.15.37:0/1031912952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7fa4f27fc640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4f41625a0 0x7fa4f40cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f62e5c65640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e0167840 0x7f62e009db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f62e5c65640 1 -- 172.21.15.37:0/2394549067 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e0155fa0 con 0x7f62e009e0d0 2024-04-26T20:21:55.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f3f7ffff640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f80167840 0x7f3f801474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f3f7f7fe640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 0x7f3f800c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.468+0000 7f424d164640 1 -- start start 2024-04-26T20:21:55.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f9ca1e45640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c9400c0d0 0x7f9c9404bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f3f7effd640 1 -- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 msgr2=0x7f3f800c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f3f7effd640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 0x7f3f800c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f3f7effd640 1 -- 172.21.15.37:0/1674363528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f80167840 msgr2=0x7f3f801474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f3f7effd640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f80167840 0x7f3f801474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f9ca2646640 1 -- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c9400c0d0 msgr2=0x7f9c9404bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f3f7effd640 1 -- 172.21.15.37:0/1674363528 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8014a230 con 0x7f3f80163690 2024-04-26T20:21:55.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f9ca2646640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c9400c0d0 0x7f9c9404bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f9ca2646640 1 -- 172.21.15.37:0/1031912952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c9404f0b0 msgr2=0x7f9c940ddc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f9ca2646640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c9404f0b0 0x7f9c940ddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f9ca2646640 1 -- 172.21.15.37:0/1031912952 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9404dbf0 con 0x7f9c940546c0 2024-04-26T20:21:55.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f9ca1e45640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c9400c0d0 0x7f9c9404bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:55.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f3f7f7fe640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f800ce860 0x7f3f800c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:55.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7ff93cd49640 1 -- 172.21.15.37:0/884124353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9381fa150 msgr2=0x7ff93815c470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:21:55.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7ff93cd49640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9381fa150 0x7ff93815c470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7ff93cd49640 1 -- 172.21.15.37:0/884124353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff938156010 msgr2=0x7ff938159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7ff93cd49640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff938156010 0x7ff938159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7ff93cd49640 1 -- 172.21.15.37:0/884124353 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93815d580 con 0x7ff93815c9b0 2024-04-26T20:21:55.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7ff937fff640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff938156010 0x7ff938159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:55.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7fa4f37fe640 1 -- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4f41625a0 msgr2=0x7fa4f40cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7fa4f37fe640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4f41625a0 0x7fa4f40cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7fa4f37fe640 1 -- 172.21.15.37:0/3157646589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f40b4f70 msgr2=0x7fa4f40cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7fa4f37fe640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f40b4f70 0x7fa4f40cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7fa4f37fe640 1 -- 172.21.15.37:0/3157646589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f40b3200 con 0x7fa4f4162980 2024-04-26T20:21:55.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7fa4f2ffd640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f40b4f70 0x7fa4f40cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:55.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f62e4c63640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e0167840 0x7f62e009db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:55.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f32f0796640 1 Processor -- start 2024-04-26T20:21:55.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f3f7effd640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f80163690 0x7f3f8009caa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009870 tx=0x7f3f740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f9ca2646640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 0x7f9c940de170 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c051930 tx=0x7f9c9c069ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f32f0796640 1 -- start start 2024-04-26T20:21:55.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7ff93cd49640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff93815c9b0 0x7ff9381fcc30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff9380ce970 tx=0x7ff93000dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7fa4f37fe640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4f4162980 0x7fa4f406ccf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec030000 tx=0x7fa4ec004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f62e5c65640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e009e0d0 0x7f62e009ba50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c420 tx=0x7f62d000c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.469+0000 7f9c927fc640 1 -- 172.21.15.37:0/1031912952 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c092070 con 0x7f9c940546c0 2024-04-26T20:21:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7f9c927fc640 1 -- 172.21.15.37:0/1031912952 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9c097660 con 0x7f9c940546c0 2024-04-26T20:21:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7f9c927fc640 1 -- 172.21.15.37:0/1031912952 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c097a10 con 0x7f9c940546c0 2024-04-26T20:21:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7f9ca40a9640 1 -- 172.21.15.37:0/1031912952 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9404ded0 con 0x7f9c940546c0 2024-04-26T20:21:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7f424d164640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42380039b0 0x7f4240036d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7f424d164640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42400e89c0 0x7f424002be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7f9ca40a9640 1 -- 172.21.15.37:0/1031912952 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9404e490 con 0x7f9c940546c0 2024-04-26T20:21:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7f32f0796640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32e0050500 0x7f32e00d3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32f0796640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00dfd40 0x7f32e0007f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32f0796640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32e00e0120 0x7f32e00084c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32f0796640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f32e00092f0 con 0x7f32e0050500 2024-04-26T20:21:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32f0796640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f32e0055550 con 0x7f32e00e0120 2024-04-26T20:21:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32f0796640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f32e0009470 con 0x7f32e00dfd40 2024-04-26T20:21:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32ee532640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32e0050500 0x7f32e00d3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32eed33640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32e00e0120 0x7f32e00084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32ee532640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32e0050500 0x7f32e00d3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59726/0 (socket says 172.21.15.37:59726) 2024-04-26T20:21:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32eed33640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32e00e0120 0x7f32e00084c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:43244/0 (socket says 172.21.15.37:43244) 2024-04-26T20:21:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32eed33640 1 -- 172.21.15.37:0/1172087317 learned_addr learned my addr 172.21.15.37:0/1172087317 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32edd31640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00dfd40 0x7f32e0007f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f62d5ffb640 1 -- 172.21.15.37:0/2394549067 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d00052d0 con 0x7f62e009e0d0 2024-04-26T20:21:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7f3f6bfff640 1 -- 172.21.15.37:0/1674363528 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74041020 con 0x7f3f80163690 2024-04-26T20:21:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7f3f6bfff640 1 -- 172.21.15.37:0/1674363528 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f74030c70 con 0x7f3f80163690 2024-04-26T20:21:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7f3f6bfff640 1 -- 172.21.15.37:0/1674363528 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74037040 con 0x7f3f80163690 2024-04-26T20:21:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f3f8605b640 1 -- 172.21.15.37:0/1674363528 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f800ca6b0 con 0x7f3f80163690 2024-04-26T20:21:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7fa4db7fe640 1 -- 172.21.15.37:0/3157646589 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec041020 con 0x7fa4f4162980 2024-04-26T20:21:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7fa4db7fe640 1 -- 172.21.15.37:0/3157646589 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec037040 con 0x7fa4f4162980 2024-04-26T20:21:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7fa4db7fe640 1 -- 172.21.15.37:0/3157646589 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec036050 con 0x7fa4f4162980 2024-04-26T20:21:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f40b2160 con 0x7fa4f4162980 2024-04-26T20:21:55.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f40b2720 con 0x7fa4f4162980 2024-04-26T20:21:55.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7fd3a4447640 1 Processor -- start 2024-04-26T20:21:55.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7fd3a4447640 1 -- start start 2024-04-26T20:21:55.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f9ca40a9640 1 -- 172.21.15.37:0/1031912952 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c940053c0 con 0x7f9c940546c0 2024-04-26T20:21:55.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f3f8605b640 1 -- 172.21.15.37:0/1674363528 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f800ca950 con 0x7f3f80163690 2024-04-26T20:21:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f62d5ffb640 1 -- 172.21.15.37:0/2394549067 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d0005490 con 0x7f62e009e0d0 2024-04-26T20:21:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f32eed33640 1 -- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00dfd40 msgr2=0x7f32e0007f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f32eed33640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00dfd40 0x7f32e0007f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f32eed33640 1 -- 172.21.15.37:0/1172087317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32e0050500 msgr2=0x7f32e00d3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f32eed33640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32e0050500 0x7f32e00d3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f32eed33640 1 -- 172.21.15.37:0/1172087317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e00d4250 con 0x7f32e00e0120 2024-04-26T20:21:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f32ee532640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32e0050500 0x7f32e00d3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f32eed33640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32e00e0120 0x7f32e00084c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32d800b2c0 tx=0x7f32d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7ff934ff9640 1 -- 172.21.15.37:0/884124353 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930018070 con 0x7ff93815c9b0 2024-04-26T20:21:55.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7ff934ff9640 1 -- 172.21.15.37:0/884124353 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff930003d80 con 0x7ff93815c9b0 2024-04-26T20:21:55.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7ff934ff9640 1 -- 172.21.15.37:0/884124353 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9300052b0 con 0x7ff93815c9b0 2024-04-26T20:21:55.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7ff93d54a640 1 -- 172.21.15.37:0/884124353 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9381fa5d0 con 0x7ff93815c9b0 2024-04-26T20:21:55.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.470+0000 7ff93d54a640 1 -- 172.21.15.37:0/884124353 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93815aa10 con 0x7ff93815c9b0 2024-04-26T20:21:55.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7ff93d54a640 1 -- 172.21.15.37:0/884124353 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff918001030 con 0x7ff93815c9b0 2024-04-26T20:21:55.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f62d5ffb640 1 -- 172.21.15.37:0/2394549067 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0007260 con 0x7f62e009e0d0 2024-04-26T20:21:55.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f62e76c8640 1 -- 172.21.15.37:0/2394549067 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e0166350 con 0x7f62e009e0d0 2024-04-26T20:21:55.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f424d164640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f424002b470 0x7f424004a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f424d164640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4240054940 con 0x7f424002b470 2024-04-26T20:21:55.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f424d164640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42400547c0 con 0x7f42400e89c0 2024-04-26T20:21:55.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f424d164640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f424004f550 con 0x7f42380039b0 2024-04-26T20:21:55.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f42477fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42400e89c0 0x7f424002be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7f42477fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42400e89c0 0x7f424002be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:43246/0 (socket says 172.21.15.37:43246) 2024-04-26T20:21:55.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f42477fe640 1 -- 172.21.15.37:0/2188849007 learned_addr learned my addr 172.21.15.37:0/2188849007 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa4f4076890 con 0x7fa4f4162980 2024-04-26T20:21:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7fd3a4447640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd394037ba0 0x7fd39404cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7fd3a4447640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39404f860 0x7fd3940db360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7fd3a4447640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3940e32c0 0x7fd3940dc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7fd3a4447640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd394053350 con 0x7fd3940e32c0 2024-04-26T20:21:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7fd3a4447640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd394054210 con 0x7fd394037ba0 2024-04-26T20:21:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.471+0000 7fd3a4447640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd394054730 con 0x7fd39404f860 2024-04-26T20:21:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7fd3a21e3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd394037ba0 0x7fd39404cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7fd3a29e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3940e32c0 0x7fd3940dc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7fd3a21e3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd394037ba0 0x7fd39404cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:43260/0 (socket says 172.21.15.37:43260) 2024-04-26T20:21:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7fd3a21e3640 1 -- 172.21.15.37:0/3794778466 learned_addr learned my addr 172.21.15.37:0/3794778466 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7fd3a19e2640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39404f860 0x7fd3940db360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f32edd31640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32e00dfd40 0x7f32e0007f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:55.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f62e76c8640 1 -- 172.21.15.37:0/2394549067 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e01645c0 con 0x7f62e009e0d0 2024-04-26T20:21:55.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f4247fff640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42380039b0 0x7f4240036d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f42477fe640 1 -- 172.21.15.37:0/2188849007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42380039b0 msgr2=0x7f4240036d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f42477fe640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42380039b0 0x7f4240036d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f42477fe640 1 -- 172.21.15.37:0/2188849007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f424002b470 msgr2=0x7f424004a3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T20:21:55.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f42477fe640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f424002b470 0x7f424004a3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f3f8605b640 1 -- 172.21.15.37:0/1674363528 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f800ca500 con 0x7f3f80163690 2024-04-26T20:21:55.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f42477fe640 1 -- 172.21.15.37:0/2188849007 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4240188d90 con 0x7f42400e89c0 2024-04-26T20:21:55.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7f42477fe640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42400e89c0 0x7f424002be20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4234011ca0 tx=0x7f423400f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7fd3a29e4640 1 -- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39404f860 msgr2=0x7fd3940db360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7fd3a29e4640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39404f860 0x7fd3940db360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7fd3a29e4640 1 -- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd394037ba0 msgr2=0x7fd39404cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7fd3a29e4640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd394037ba0 0x7fd39404cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.472+0000 7fd3a29e4640 1 -- 172.21.15.37:0/3794778466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3940db8a0 con 0x7fd3940e32c0 2024-04-26T20:21:55.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7fa4db7fe640 1 -- 172.21.15.37:0/3157646589 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4ec007990 con 0x7fa4f4162980 2024-04-26T20:21:55.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7f32deffd640 1 -- 172.21.15.37:0/1172087317 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8036040 con 0x7f32e00e0120 2024-04-26T20:21:55.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7fd3a29e4640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3940e32c0 0x7fd3940dc500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd38c031000 tx=0x7fd38c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7f32deffd640 1 -- 172.21.15.37:0/1172087317 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d80453f0 con 0x7f32e00e0120 2024-04-26T20:21:55.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7f32f0796640 1 -- 172.21.15.37:0/1172087317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e0049790 con 0x7f32e00e0120 2024-04-26T20:21:55.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7f32deffd640 1 -- 172.21.15.37:0/1172087317 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d80457c0 con 0x7f32e00e0120 2024-04-26T20:21:55.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7f32f0796640 1 -- 172.21.15.37:0/1172087317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e0049d50 con 0x7f32e00e0120 2024-04-26T20:21:55.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7f62e76c8640 1 -- 172.21.15.37:0/2394549067 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62e009e930 con 0x7f62e009e0d0 2024-04-26T20:21:55.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7fd392ffd640 1 -- 172.21.15.37:0/3794778466 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c037050 con 0x7fd3940e32c0 2024-04-26T20:21:55.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7fa4db7fe640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa4e006c940 0x7fa4e006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7f32f0796640 1 -- 172.21.15.37:0/1172087317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32e00057d0 con 0x7f32e00e0120 2024-04-26T20:21:55.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7f4244ff9640 1 -- 172.21.15.37:0/2188849007 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423400fca0 con 0x7f42400e89c0 2024-04-26T20:21:55.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7f4244ff9640 1 -- 172.21.15.37:0/2188849007 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f423400fe40 con 0x7f42400e89c0 2024-04-26T20:21:55.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7f4244ff9640 1 -- 172.21.15.37:0/2188849007 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234014680 con 0x7f42400e89c0 2024-04-26T20:21:55.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7f424d164640 1 -- 172.21.15.37:0/2188849007 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4240189070 con 0x7f42400e89c0 2024-04-26T20:21:55.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7fd3a4447640 1 -- 172.21.15.37:0/3794778466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39404bd60 con 0x7fd3940e32c0 2024-04-26T20:21:55.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.473+0000 7fd3a4447640 1 -- 172.21.15.37:0/3794778466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39404c320 con 0x7fd3940e32c0 2024-04-26T20:21:55.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7fd392ffd640 1 -- 172.21.15.37:0/3794778466 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38c038040 con 0x7fd3940e32c0 2024-04-26T20:21:55.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7f424d164640 1 -- 172.21.15.37:0/2188849007 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4240189630 con 0x7f42400e89c0 2024-04-26T20:21:55.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7fa4db7fe640 1 -- 172.21.15.37:0/3157646589 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4ec0ac4e0 con 0x7fa4f4162980 2024-04-26T20:21:55.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7fa4db7fe640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] conn(0x7fa4e0074ae0 0x7fa4e0076f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7fa4db7fe640 1 -- 172.21.15.37:0/3157646589 --> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4ec031600 con 0x7fa4e0074ae0 2024-04-26T20:21:55.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7fd392ffd640 1 -- 172.21.15.37:0/3794778466 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c004450 con 0x7fd3940e32c0 2024-04-26T20:21:55.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7fd3a4447640 1 -- 172.21.15.37:0/3794778466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd384001030 con 0x7fd3940e32c0 2024-04-26T20:21:55.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7f32deffd640 1 -- 172.21.15.37:0/1172087317 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32d804c060 con 0x7f32e00e0120 2024-04-26T20:21:55.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7f3f6bfff640 1 -- 172.21.15.37:0/1674363528 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f74044020 con 0x7f3f80163690 2024-04-26T20:21:55.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7fa4db7fe640 1 -- 172.21.15.37:0/3157646589 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa4ec0b3050 con 0x7fa4f4162980 2024-04-26T20:21:55.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7ff934ff9640 1 -- 172.21.15.37:0/884124353 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff93001b070 con 0x7ff93815c9b0 2024-04-26T20:21:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.474+0000 7f62d5ffb640 1 -- 172.21.15.37:0/2394549067 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62d0007400 con 0x7f62e009e0d0 2024-04-26T20:21:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7f62d5ffb640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f62c406c9c0 0x7f62c406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7f62d5ffb640 1 -- 172.21.15.37:0/2394549067 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f62d008aeb0 con 0x7f62e009e0d0 2024-04-26T20:21:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f32deffd640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f32cc06c8f0 0x7f32cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7f62d5ffb640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] conn(0x7f62c4074be0 0x7f62c4077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7f62d5ffb640 1 -- 172.21.15.37:0/2394549067 --> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62d0006e70 con 0x7f62c4074be0 2024-04-26T20:21:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7f3f6bfff640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3f6406c9f0 0x7f3f6406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7f3f6bfff640 1 -- 172.21.15.37:0/1674363528 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f740ac400 con 0x7f3f80163690 2024-04-26T20:21:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f3f6bfff640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] conn(0x7f3f64074b90 0x7f3f64076fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f3f6bfff640 1 -- 172.21.15.37:0/1674363528 --> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f7402be50 con 0x7f3f64074b90 2024-04-26T20:21:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f3f6bfff640 1 -- 172.21.15.37:0/1674363528 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3f740b3050 con 0x7f3f80163690 2024-04-26T20:21:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7fa4f2ffd640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa4e006c940 0x7fa4e006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7f62d5ffb640 1 -- 172.21.15.37:0/2394549067 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f62d00904c0 con 0x7f62e009e0d0 2024-04-26T20:21:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f32deffd640 1 -- 172.21.15.37:0/1172087317 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f32d80ae780 con 0x7f32e00e0120 2024-04-26T20:21:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f62e4c63640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] conn(0x7f62c4074be0 0x7f62c4077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7ff934ff9640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff92406c9f0 0x7ff92406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7ff934ff9640 1 -- 172.21.15.37:0/884124353 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff93008baa0 con 0x7ff93815c9b0 2024-04-26T20:21:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7ff934ff9640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] conn(0x7ff924074c10 0x7ff924077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7ff934ff9640 1 -- 172.21.15.37:0/884124353 --> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9240776e0 con 0x7ff924074c10 2024-04-26T20:21:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.475+0000 7ff934ff9640 1 -- 172.21.15.37:0/884124353 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff93008be90 con 0x7ff93815c9b0 2024-04-26T20:21:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7ff9377fe640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] conn(0x7ff924074c10 0x7ff924077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7fa4f27fc640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] conn(0x7fa4e0074ae0 0x7fa4e0076f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f32deffd640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] conn(0x7f32cc074a90 0x7f32cc076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f32ee532640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f32cc06c8f0 0x7f32cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f3f7ffff640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] conn(0x7f3f64074b90 0x7f3f64076fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f3f7f7fe640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3f6406c9f0 0x7f3f6406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7ff937fff640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff92406c9f0 0x7ff92406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f62e5464640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f62c406c9c0 0x7f62c406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f62e4c63640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] conn(0x7f62c4074be0 0x7f62c4077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f4244ff9640 1 -- 172.21.15.37:0/2188849007 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4234012040 con 0x7f42400e89c0 2024-04-26T20:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f42327fc640 1 -- 172.21.15.37:0/2188849007 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4240189a10 con 0x7f42400e89c0 2024-04-26T20:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f3f7ffff640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] conn(0x7f3f64074b90 0x7f3f64076fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7fa4f27fc640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] conn(0x7fa4e0074ae0 0x7fa4e0076f00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7fd392ffd640 1 -- 172.21.15.37:0/3794778466 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd38c003610 con 0x7fd3940e32c0 2024-04-26T20:21:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.476+0000 7f32deffd640 1 -- 172.21.15.37:0/1172087317 --> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32d8064ec0 con 0x7f32cc074a90 2024-04-26T20:21:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f32deffd640 1 -- 172.21.15.37:0/1172087317 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f32d80b5050 con 0x7f32e00e0120 2024-04-26T20:21:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f32edd31640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] conn(0x7f32cc074a90 0x7f32cc076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f62e5464640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f62c406c9c0 0x7f62c406ee80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009870 tx=0x7f62dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f9c927fc640 1 -- 172.21.15.37:0/1031912952 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c9c06f050 con 0x7f9c940546c0 2024-04-26T20:21:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f9c927fc640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9c8006c8f0 0x7f9c8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7fd392ffd640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd38006c9c0 0x7fd38006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7ff937fff640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff92406c9f0 0x7ff92406eeb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff928032000 tx=0x7ff928037040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f4244ff9640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f422c06c940 0x7f422c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f32edd31640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] conn(0x7f32cc074a90 0x7f32cc076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f9c927fc640 1 -- 172.21.15.37:0/1031912952 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9c9c108ff0 con 0x7f9c940546c0 2024-04-26T20:21:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.477+0000 7f9c927fc640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] conn(0x7f9c80074b70 0x7f9c80076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f9c927fc640 1 -- 172.21.15.37:0/1031912952 --> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c9c09ee80 con 0x7f9c80074b70 2024-04-26T20:21:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f3f7f7fe640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3f6406c9f0 0x7f3f6406eeb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f78007fd0 tx=0x7f3f78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f9c927fc640 1 -- 172.21.15.37:0/1031912952 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9c9c1093e0 con 0x7f9c940546c0 2024-04-26T20:21:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7fd392ffd640 1 -- 172.21.15.37:0/3794778466 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd38c0acb00 con 0x7fd3940e32c0 2024-04-26T20:21:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7fd3a21e3640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd38006c9c0 0x7fd38006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f62d5ffb640 1 -- 172.21.15.37:0/2394549067 <== osd.2 v2:172.21.15.37:6816/2890277493 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62d0006e70 con 0x7f62c4074be0 2024-04-26T20:21:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f3f6bfff640 1 -- 172.21.15.37:0/1674363528 <== osd.5 v2:172.21.15.134:6808/1335053617 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f7402be50 con 0x7f3f64074b90 2024-04-26T20:21:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7fa4f2ffd640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa4e006c940 0x7fa4e006ee00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4004350 tx=0x7fa4e4007ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7fd392ffd640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] conn(0x7fd380074be0 0x7fd380077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7fd392ffd640 1 -- 172.21.15.37:0/3794778466 --> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd38c008eb0 con 0x7fd380074be0 2024-04-26T20:21:55.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f9ca1644640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] conn(0x7f9c80074b70 0x7f9c80076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7fd392ffd640 1 -- 172.21.15.37:0/3794778466 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd38c0acef0 con 0x7fd3940e32c0 2024-04-26T20:21:55.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f9ca1e45640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9c8006c8f0 0x7f9c8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7ff9377fe640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] conn(0x7ff924074c10 0x7ff924077030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f4244ff9640 1 -- 172.21.15.37:0/2188849007 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f423408c270 con 0x7f42400e89c0 2024-04-26T20:21:55.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f4244ff9640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] conn(0x7f422c074b60 0x7f422c076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f4247fff640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f422c06c940 0x7f422c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f424c963640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] conn(0x7f422c074b60 0x7f422c076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f424c963640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] conn(0x7f422c074b60 0x7f422c076f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.479+0000 7f9ca1e45640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9c8006c8f0 0x7f9c8006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9c98034000 tx=0x7f9c9802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.479+0000 7f9ca1644640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] conn(0x7f9c80074b70 0x7f9c80076f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.479+0000 7f4247fff640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f422c06c940 0x7f422c06ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f423c033000 tx=0x7f423c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.478+0000 7f32ee532640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f32cc06c8f0 0x7f32cc06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f32e4001bc0 tx=0x7f32e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.479+0000 7f4244ff9640 1 -- 172.21.15.37:0/2188849007 --> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4234025ec0 con 0x7f422c074b60 2024-04-26T20:21:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.479+0000 7f32deffd640 1 -- 172.21.15.37:0/1172087317 <== osd.0 v2:172.21.15.37:6808/2482772026 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32d8064ec0 con 0x7f32cc074a90 2024-04-26T20:21:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.479+0000 7f4244ff9640 1 -- 172.21.15.37:0/2188849007 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4234093050 con 0x7f42400e89c0 2024-04-26T20:21:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.479+0000 7fd3a19e2640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] conn(0x7fd380074be0 0x7fd380077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.479+0000 7fa4db7fe640 1 -- 172.21.15.37:0/3157646589 <== osd.6 v2:172.21.15.134:6816/1715563323 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4ec031600 con 0x7fa4e0074ae0 2024-04-26T20:21:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.479+0000 7fd3a21e3640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd38006c9c0 0x7fd38006ee80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd398001eb0 tx=0x7fd398001750 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.479+0000 7ff934ff9640 1 -- 172.21.15.37:0/884124353 <== osd.1 v2:172.21.15.37:6800/2029719416 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9240776e0 con 0x7ff924074c10 2024-04-26T20:21:55.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.480+0000 7f4244ff9640 1 -- 172.21.15.37:0/2188849007 <== osd.3 v2:172.21.15.37:6819/2156822987 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4234025ec0 con 0x7f422c074b60 2024-04-26T20:21:55.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.480+0000 7f9c927fc640 1 -- 172.21.15.37:0/1031912952 <== osd.4 v2:172.21.15.134:6824/2837585248 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c9c09ee80 con 0x7f9c80074b70 2024-04-26T20:21:55.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.480+0000 7fd3a19e2640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] conn(0x7fd380074be0 0x7fd380077000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.481+0000 7fd392ffd640 1 -- 172.21.15.37:0/3794778466 <== osd.7 v2:172.21.15.134:6800/4008114040 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd38c008eb0 con 0x7fd380074be0 2024-04-26T20:21:55.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.510+0000 7f62e76c8640 1 -- 172.21.15.37:0/2394549067 --> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62e006a360 con 0x7f62c4074be0 2024-04-26T20:21:55.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.511+0000 7f62d5ffb640 1 -- 172.21.15.37:0/2394549067 <== osd.2 v2:172.21.15.37:6816/2890277493 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f62e006a360 con 0x7f62c4074be0 2024-04-26T20:21:55.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.511+0000 7f3f8605b640 1 -- 172.21.15.37:0/1674363528 --> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3f8006a360 con 0x7f3f64074b90 2024-04-26T20:21:55.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.512+0000 7f62c37fe640 1 -- 172.21.15.37:0/2394549067 >> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] conn(0x7f62c4074be0 msgr2=0x7f62c4077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.512+0000 7f62c37fe640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] conn(0x7f62c4074be0 0x7f62c4077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.512+0000 7f62c37fe640 1 -- 172.21.15.37:0/2394549067 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f62c406c9c0 msgr2=0x7f62c406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.512+0000 7f3f6bfff640 1 -- 172.21.15.37:0/1674363528 <== osd.5 v2:172.21.15.134:6808/1335053617 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3f8006a360 con 0x7f3f64074b90 2024-04-26T20:21:55.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.512+0000 7f62c37fe640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f62c406c9c0 0x7f62c406ee80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009870 tx=0x7f62dc008040 comp rx=0 tx=0).stop 2024-04-26T20:21:55.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.512+0000 7f62c37fe640 1 -- 172.21.15.37:0/2394549067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e009e0d0 msgr2=0x7f62e009ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.512+0000 7f62c37fe640 1 --2- 172.21.15.37:0/2394549067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e009e0d0 0x7f62e009ba50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c420 tx=0x7f62d000c8e0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.512+0000 7f32f0796640 1 -- 172.21.15.37:0/1172087317 --> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32e004a210 con 0x7f32cc074a90 2024-04-26T20:21:55.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.512+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 --> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa4f40b4d20 con 0x7fa4e0074ae0 2024-04-26T20:21:55.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f62e5c65640 1 -- 172.21.15.37:0/2394549067 reap_dead start 2024-04-26T20:21:55.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f62c37fe640 1 -- 172.21.15.37:0/2394549067 shutdown_connections 2024-04-26T20:21:55.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f62c37fe640 1 -- 172.21.15.37:0/2394549067 >> 172.21.15.37:0/2394549067 conn(0x7f62e00d0c90 msgr2=0x7f62e0166fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f62c37fe640 1 -- 172.21.15.37:0/2394549067 shutdown_connections 2024-04-26T20:21:55.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f3f8605b640 1 -- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] conn(0x7f3f64074b90 msgr2=0x7f3f64076fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f62c37fe640 1 -- 172.21.15.37:0/2394549067 wait complete. 2024-04-26T20:21:55.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f3f8605b640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] conn(0x7f3f64074b90 0x7f3f64076fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f3f8605b640 1 -- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3f6406c9f0 msgr2=0x7f3f6406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f3f8605b640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3f6406c9f0 0x7f3f6406eeb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f78007fd0 tx=0x7f3f78009040 comp rx=0 tx=0).stop 2024-04-26T20:21:55.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f3f8605b640 1 -- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f80163690 msgr2=0x7f3f8009caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f3f8605b640 1 --2- 172.21.15.37:0/1674363528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f80163690 0x7f3f8009caa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009870 tx=0x7f3f740046c0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7fa4db7fe640 1 -- 172.21.15.37:0/3157646589 <== osd.6 v2:172.21.15.134:6816/1715563323 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa4f40b4d20 con 0x7fa4e0074ae0 2024-04-26T20:21:55.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f32deffd640 1 -- 172.21.15.37:0/1172087317 <== osd.0 v2:172.21.15.37:6808/2482772026 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f32e004a210 con 0x7f32cc074a90 2024-04-26T20:21:55.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f3f7ffff640 1 -- 172.21.15.37:0/1674363528 reap_dead start 2024-04-26T20:21:55.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f9ca40a9640 1 -- 172.21.15.37:0/1031912952 --> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c9404e660 con 0x7f9c80074b70 2024-04-26T20:21:55.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f3f8605b640 1 -- 172.21.15.37:0/1674363528 shutdown_connections 2024-04-26T20:21:55.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7f3f8605b640 1 -- 172.21.15.37:0/1674363528 >> 172.21.15.37:0/1674363528 conn(0x7f3f800d0c90 msgr2=0x7f3f800cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.513+0000 7fd3a4447640 1 -- 172.21.15.37:0/3794778466 --> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd384002dd0 con 0x7fd380074be0 2024-04-26T20:21:55.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f32f0796640 1 -- 172.21.15.37:0/1172087317 >> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] conn(0x7f32cc074a90 msgr2=0x7f32cc076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f32f0796640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] conn(0x7f32cc074a90 0x7f32cc076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f3f8605b640 1 -- 172.21.15.37:0/1674363528 shutdown_connections 2024-04-26T20:21:55.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f3f8605b640 1 -- 172.21.15.37:0/1674363528 wait complete. 2024-04-26T20:21:55.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f32f0796640 1 -- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f32cc06c8f0 msgr2=0x7f32cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f32f0796640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f32cc06c8f0 0x7f32cc06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f32e4001bc0 tx=0x7f32e4009040 comp rx=0 tx=0).stop 2024-04-26T20:21:55.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f32f0796640 1 -- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32e00e0120 msgr2=0x7f32e00084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f32f0796640 1 --2- 172.21.15.37:0/1172087317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32e00e0120 0x7f32e00084c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32d800b2c0 tx=0x7f32d80046c0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f32eed33640 1 -- 172.21.15.37:0/1172087317 reap_dead start 2024-04-26T20:21:55.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f32f0796640 1 -- 172.21.15.37:0/1172087317 shutdown_connections 2024-04-26T20:21:55.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f32f0796640 1 -- 172.21.15.37:0/1172087317 >> 172.21.15.37:0/1172087317 conn(0x7f32e0055bc0 msgr2=0x7f32e0036b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f42327fc640 1 -- 172.21.15.37:0/2188849007 --> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4240005390 con 0x7f422c074b60 2024-04-26T20:21:55.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f9c927fc640 1 -- 172.21.15.37:0/1031912952 <== osd.4 v2:172.21.15.134:6824/2837585248 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9c9404e660 con 0x7f9c80074b70 2024-04-26T20:21:55.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7fd392ffd640 1 -- 172.21.15.37:0/3794778466 <== osd.7 v2:172.21.15.134:6800/4008114040 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd384002dd0 con 0x7fd380074be0 2024-04-26T20:21:55.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7ff93d54a640 1 -- 172.21.15.37:0/884124353 --> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff918002d50 con 0x7ff924074c10 2024-04-26T20:21:55.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] conn(0x7fa4e0074ae0 msgr2=0x7fa4e0076f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7fa4f3fff640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] conn(0x7fa4e0074ae0 0x7fa4e0076f00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f4244ff9640 1 -- 172.21.15.37:0/2188849007 <== osd.3 v2:172.21.15.37:6819/2156822987 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4240005390 con 0x7f422c074b60 2024-04-26T20:21:55.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f9ca40a9640 1 -- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] conn(0x7f9c80074b70 msgr2=0x7f9c80076f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7f9ca40a9640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] conn(0x7f9c80074b70 0x7f9c80076f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7ff934ff9640 1 -- 172.21.15.37:0/884124353 <== osd.1 v2:172.21.15.37:6800/2029719416 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff918002d50 con 0x7ff924074c10 2024-04-26T20:21:55.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.514+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa4e006c940 msgr2=0x7fa4e006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fa4f3fff640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa4e006c940 0x7fa4e006ee00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4004350 tx=0x7fa4e4007ef0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4f4162980 msgr2=0x7fa4f406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fa4f3fff640 1 --2- 172.21.15.37:0/3157646589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4f4162980 0x7fa4f406ccf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec030000 tx=0x7fa4ec004a40 comp rx=0 tx=0).stop 2024-04-26T20:21:55.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fd3a4447640 1 -- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] conn(0x7fd380074be0 msgr2=0x7fd380077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fa4f37fe640 1 -- 172.21.15.37:0/3157646589 reap_dead start 2024-04-26T20:21:55.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fd3a4447640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] conn(0x7fd380074be0 0x7fd380077000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f32f0796640 1 -- 172.21.15.37:0/1172087317 shutdown_connections 2024-04-26T20:21:55.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f9ca40a9640 1 -- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9c8006c8f0 msgr2=0x7f9c8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f9ca40a9640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9c8006c8f0 0x7f9c8006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9c98034000 tx=0x7f9c9802e000 comp rx=0 tx=0).stop 2024-04-26T20:21:55.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f9ca40a9640 1 -- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 msgr2=0x7f9c940de170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f9ca40a9640 1 --2- 172.21.15.37:0/1031912952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c940546c0 0x7f9c940de170 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c051930 tx=0x7f9c9c069ec0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f9ca2646640 1 -- 172.21.15.37:0/1031912952 reap_dead start 2024-04-26T20:21:55.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f9ca40a9640 1 -- 172.21.15.37:0/1031912952 shutdown_connections 2024-04-26T20:21:55.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f9ca40a9640 1 -- 172.21.15.37:0/1031912952 >> 172.21.15.37:0/1031912952 conn(0x7f9c94009fe0 msgr2=0x7f9c940d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 shutdown_connections 2024-04-26T20:21:55.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 >> 172.21.15.37:0/3157646589 conn(0x7fa4f40d4fa0 msgr2=0x7fa4f415df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f42327fc640 1 -- 172.21.15.37:0/2188849007 >> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] conn(0x7f422c074b60 msgr2=0x7f422c076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f42327fc640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] conn(0x7f422c074b60 0x7f422c076f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 shutdown_connections 2024-04-26T20:21:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f32f0796640 1 -- 172.21.15.37:0/1172087317 wait complete. 2024-04-26T20:21:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fa4f3fff640 1 -- 172.21.15.37:0/3157646589 wait complete. 2024-04-26T20:21:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7ff91e7fc640 1 -- 172.21.15.37:0/884124353 >> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] conn(0x7ff924074c10 msgr2=0x7ff924077030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7ff91e7fc640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] conn(0x7ff924074c10 0x7ff924077030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f9ca40a9640 1 -- 172.21.15.37:0/1031912952 shutdown_connections 2024-04-26T20:21:55.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fd3a4447640 1 -- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd38006c9c0 msgr2=0x7fd38006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fd3a4447640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd38006c9c0 0x7fd38006ee80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd398001eb0 tx=0x7fd398001750 comp rx=0 tx=0).stop 2024-04-26T20:21:55.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fd3a4447640 1 -- 172.21.15.37:0/3794778466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3940e32c0 msgr2=0x7fd3940dc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fd3a4447640 1 --2- 172.21.15.37:0/3794778466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3940e32c0 0x7fd3940dc500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd38c031000 tx=0x7fd38c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fd3a29e4640 1 -- 172.21.15.37:0/3794778466 reap_dead start 2024-04-26T20:21:55.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f9ca40a9640 1 -- 172.21.15.37:0/1031912952 wait complete. 2024-04-26T20:21:55.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fd3a4447640 1 -- 172.21.15.37:0/3794778466 shutdown_connections 2024-04-26T20:21:55.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7fd3a4447640 1 -- 172.21.15.37:0/3794778466 >> 172.21.15.37:0/3794778466 conn(0x7fd394009fa0 msgr2=0x7fd3940057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f42327fc640 1 -- 172.21.15.37:0/2188849007 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f422c06c940 msgr2=0x7f422c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f42327fc640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f422c06c940 0x7f422c06ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f423c033000 tx=0x7f423c008030 comp rx=0 tx=0).stop 2024-04-26T20:21:55.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f42327fc640 1 -- 172.21.15.37:0/2188849007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42400e89c0 msgr2=0x7f424002be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7f42327fc640 1 --2- 172.21.15.37:0/2188849007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42400e89c0 0x7f424002be20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4234011ca0 tx=0x7f423400f610 comp rx=0 tx=0).stop 2024-04-26T20:21:55.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7ff91e7fc640 1 -- 172.21.15.37:0/884124353 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff92406c9f0 msgr2=0x7ff92406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7ff91e7fc640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff92406c9f0 0x7ff92406eeb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff928032000 tx=0x7ff928037040 comp rx=0 tx=0).stop 2024-04-26T20:21:55.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7ff91e7fc640 1 -- 172.21.15.37:0/884124353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff93815c9b0 msgr2=0x7ff9381fcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.515+0000 7ff91e7fc640 1 --2- 172.21.15.37:0/884124353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff93815c9b0 0x7ff9381fcc30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff9380ce970 tx=0x7ff93000dc10 comp rx=0 tx=0).stop 2024-04-26T20:21:55.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7f424c963640 1 -- 172.21.15.37:0/2188849007 reap_dead start 2024-04-26T20:21:55.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7fd3a4447640 1 -- 172.21.15.37:0/3794778466 shutdown_connections 2024-04-26T20:21:55.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7fd3a4447640 1 -- 172.21.15.37:0/3794778466 wait complete. 2024-04-26T20:21:55.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7f42327fc640 1 -- 172.21.15.37:0/2188849007 shutdown_connections 2024-04-26T20:21:55.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7f42327fc640 1 -- 172.21.15.37:0/2188849007 >> 172.21.15.37:0/2188849007 conn(0x7f4240009fb0 msgr2=0x7f42400e9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7ff93cd49640 1 -- 172.21.15.37:0/884124353 reap_dead start 2024-04-26T20:21:55.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7f42327fc640 1 -- 172.21.15.37:0/2188849007 shutdown_connections 2024-04-26T20:21:55.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7f42327fc640 1 -- 172.21.15.37:0/2188849007 wait complete. 2024-04-26T20:21:55.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7ff91e7fc640 1 -- 172.21.15.37:0/884124353 shutdown_connections 2024-04-26T20:21:55.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7ff91e7fc640 1 -- 172.21.15.37:0/884124353 >> 172.21.15.37:0/884124353 conn(0x7ff9380b2890 msgr2=0x7ff9380b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7ff91e7fc640 1 -- 172.21.15.37:0/884124353 shutdown_connections 2024-04-26T20:21:55.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.516+0000 7ff91e7fc640 1 -- 172.21.15.37:0/884124353 wait complete. 2024-04-26T20:21:55.574 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:55.575 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T20:21:55.577 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:55.577 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T20:21:55.578 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:55.578 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T20:21:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:55834574856 2024-04-26T20:21:55.579 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T20:21:55.580 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:55.580 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T20:21:55.581 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:55.581 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T20:21:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:55.582 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T20:21:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:55834574854 2024-04-26T20:21:55.584 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T20:21:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.791+0000 7f4e5e617640 1 Processor -- start 2024-04-26T20:21:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.791+0000 7f694e595640 1 Processor -- start 2024-04-26T20:21:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.791+0000 7f4e5e617640 1 -- start start 2024-04-26T20:21:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.792+0000 7f0e2f555640 1 Processor -- start 2024-04-26T20:21:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.792+0000 7f0e2f555640 1 -- start start 2024-04-26T20:21:55.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.792+0000 7f694e595640 1 -- start start 2024-04-26T20:21:55.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.792+0000 7ff3f90c2640 1 Processor -- start 2024-04-26T20:21:55.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.792+0000 7f77900bd640 1 Processor -- start 2024-04-26T20:21:55.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.792+0000 7f5433fc9640 1 Processor -- start 2024-04-26T20:21:55.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7fab26134640 1 Processor -- start 2024-04-26T20:21:55.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7fab26134640 1 -- start start 2024-04-26T20:21:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7fab26134640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab20073a60 0x7fab2015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7fab26134640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fab2006c6e0 0x7fab20073e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7fab26134640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab20074380 con 0x7fab200b1bb0 2024-04-26T20:21:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7fab26134640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fab2006d720 con 0x7fab2006c6e0 2024-04-26T20:21:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7fab26134640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fab200cddf0 con 0x7fab20073a60 2024-04-26T20:21:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f0e2f555640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e200e30c0 0x7f0e20057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f0e2ed54640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0e20054300 0x7f0e200068d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44320/0 (socket says 172.21.15.37:44320) 2024-04-26T20:21:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f0e2ed54640 1 -- 172.21.15.37:0/359869543 learned_addr learned my addr 172.21.15.37:0/359869543 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f0e2f555640 1 -- 172.21.15.37:0/359869543 --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e200582d0 con 0x7f0e20054300 2024-04-26T20:21:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f0e2e553640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e200e30c0 0x7f0e20057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f4e5e617640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4e58072780 0x7f4e5806f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f4e5e617640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e5806e6c0 con 0x7f4e580cfa90 2024-04-26T20:21:55.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f4e5e617640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e5806e9a0 con 0x7f4e5814d2e0 2024-04-26T20:21:55.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f4e5e617640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4e5806ec80 con 0x7f4e58072780 2024-04-26T20:21:55.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f4e577fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4e580cfa90 0x7f4e58072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44278/0 (socket says 172.21.15.37:44278) 2024-04-26T20:21:55.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f4e577fe640 1 -- 172.21.15.37:0/807712707 learned_addr learned my addr 172.21.15.37:0/807712707 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:55.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 199231302 0 0) 0x7f4e5806e6c0 con 0x7f4e580cfa90 2024-04-26T20:21:55.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f4e5cbb4640 1 --2- 172.21.15.37:0/807712707 >> v2:172.21.15.134:3300/0 conn(0x7f4e58072780 0x7f4e5806f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f694e595640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69400ead00 0x7f694004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f694e595640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6940053720 con 0x7f694004ffd0 2024-04-26T20:21:55.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f694e595640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6940053a00 con 0x7f6940006ea0 2024-04-26T20:21:55.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f694e595640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6940054330 con 0x7f69400ead00 2024-04-26T20:21:55.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f694cd92640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f694004ffd0 0x7f6940037bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44286/0 (socket says 172.21.15.37:44286) 2024-04-26T20:21:55.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f694cd92640 1 -- 172.21.15.37:0/3874570685 learned_addr learned my addr 172.21.15.37:0/3874570685 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:55.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f694d593640 1 --2- 172.21.15.37:0/3874570685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69400ead00 0x7f694004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f694d593640 1 -- 172.21.15.37:0/3874570685 >> v1:172.21.15.134:6789/0 conn(0x7f6940006ea0 legacy=0x7f69400382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f694d593640 1 -- 172.21.15.37:0/3874570685 >> v1:172.21.15.37:6789/0 conn(0x7f694004ffd0 legacy=0x7f6940037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f694d593640 1 -- 172.21.15.37:0/3874570685 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6940053ce0 con 0x7f69400ead00 2024-04-26T20:21:55.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.792+0000 7ff3f90c2640 1 -- start start 2024-04-26T20:21:55.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7ff3f90c2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f4072fc0 0x7ff3f4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7ff3f90c2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3f406fe80 con 0x7ff3f415bdd0 2024-04-26T20:21:55.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7ff3f90c2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3f4070160 con 0x7ff3f406f170 2024-04-26T20:21:55.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7ff3f90c2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f40c7510 con 0x7ff3f4072fc0 2024-04-26T20:21:55.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7ff3f88c1640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff3f406f170 0x7ff3f406f5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:60770/0 (socket says 172.21.15.37:60770) 2024-04-26T20:21:55.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7ff3f88c1640 1 -- 172.21.15.37:0/3659619276 learned_addr learned my addr 172.21.15.37:0/3659619276 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:55.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7ff3f37fe640 1 --2- 172.21.15.37:0/3659619276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f4072fc0 0x7ff3f4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7ff3f2ffd640 1 -- 172.21.15.37:0/3659619276 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3676569413 0 0) 0x7ff3f4070160 con 0x7ff3f406f170 2024-04-26T20:21:55.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7ff3f2ffd640 1 -- 172.21.15.37:0/3659619276 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3dc003520 con 0x7ff3f406f170 2024-04-26T20:21:55.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.792+0000 7f77900bd640 1 -- start start 2024-04-26T20:21:55.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f77900bd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f77880cde50 0x7f77880cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f77900bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f778806c240 0x7f778806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f77900bd640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7788163c80 con 0x7f7788167350 2024-04-26T20:21:55.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f77900bd640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f7788073a90 con 0x7f77880cde50 2024-04-26T20:21:55.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f77900bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f77880d15d0 con 0x7f778806c240 2024-04-26T20:21:55.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f778de59640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f77880cde50 0x7f77880cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f778de59640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f77880cde50 0x7f77880cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34966/0 (socket says 172.21.15.37:34966) 2024-04-26T20:21:55.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f778de59640 1 -- 172.21.15.37:0/1339013501 learned_addr learned my addr 172.21.15.37:0/1339013501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f778d658640 1 --2- 172.21.15.37:0/1339013501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f778806c240 0x7f778806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.792+0000 7f5433fc9640 1 -- start start 2024-04-26T20:21:55.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f5433fc9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5424005e90 0x7f54240062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f5433fc9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 0x7f5424006ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.793+0000 7f5433fc9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5424053720 con 0x7f54240ead00 2024-04-26T20:21:55.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f5433fc9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f5424054330 con 0x7f5424005e90 2024-04-26T20:21:55.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f5433fc9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5424053a00 con 0x7f5424006810 2024-04-26T20:21:55.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f5432566640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 0x7f5424006ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f5431d65640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f54240ead00 0x7f542404fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:60774/0 (socket says 172.21.15.37:60774) 2024-04-26T20:21:55.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f5431d65640 1 -- 172.21.15.37:0/4230199400 learned_addr learned my addr 172.21.15.37:0/4230199400 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:55.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f5431564640 1 --2- 172.21.15.37:0/4230199400 >> v2:172.21.15.37:3300/0 conn(0x7f5424005e90 0x7f54240062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7fab24931640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fab2006c6e0 0x7fab20073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7fab24931640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fab2006c6e0 0x7fab20073e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:33590/0 (socket says 172.21.15.37:33590) 2024-04-26T20:21:55.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7fab24931640 1 -- 172.21.15.37:0/1409042080 learned_addr learned my addr 172.21.15.37:0/1409042080 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e40003520 con 0x7f4e580cfa90 2024-04-26T20:21:55.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.794+0000 7f47f9927640 1 Processor -- start 2024-04-26T20:21:55.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1997778682 0 0) 0x7f4e5806e9a0 con 0x7f4e5814d2e0 2024-04-26T20:21:55.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7fab25132640 1 --2- 172.21.15.37:0/1409042080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab20073a60 0x7fab2015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e5806e6c0 con 0x7f4e5814d2e0 2024-04-26T20:21:55.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f778d658640 1 -- 172.21.15.37:0/1339013501 >> v2:172.21.15.37:3300/0 conn(0x7f77880cde50 msgr2=0x7f77880cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f778d658640 1 --2- 172.21.15.37:0/1339013501 >> v2:172.21.15.37:3300/0 conn(0x7f77880cde50 0x7f77880cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f778d658640 1 -- 172.21.15.37:0/1339013501 >> v1:172.21.15.37:6789/0 conn(0x7f7788167350 legacy=0x7f77881677e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f778d658640 1 -- 172.21.15.37:0/1339013501 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77880cff10 con 0x7f778806c240 2024-04-26T20:21:55.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f0e2f555640 1 -- 172.21.15.37:0/359869543 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e20058570 con 0x7f0e2003e840 2024-04-26T20:21:55.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f0e2f555640 1 -- 172.21.15.37:0/359869543 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0e20058850 con 0x7f0e200e30c0 2024-04-26T20:21:55.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7fab24931640 1 -- 172.21.15.37:0/1409042080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab20073a60 msgr2=0x7fab2015bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7fab24931640 1 --2- 172.21.15.37:0/1409042080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab20073a60 0x7fab2015bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7fab24931640 1 -- 172.21.15.37:0/1409042080 >> v1:172.21.15.37:6789/0 conn(0x7fab200b1bb0 legacy=0x7fab200ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7fab24931640 1 -- 172.21.15.37:0/1409042080 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2006ce80 con 0x7fab2006c6e0 2024-04-26T20:21:55.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3847693176 0 0) 0x7f0e200582d0 con 0x7f0e20054300 2024-04-26T20:21:55.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e14003520 con 0x7f0e20054300 2024-04-26T20:21:55.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f5430d63640 1 -- 172.21.15.37:0/4230199400 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 907444526 0 0) 0x7f5424053720 con 0x7f54240ead00 2024-04-26T20:21:55.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f5430d63640 1 -- 172.21.15.37:0/4230199400 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5418003520 con 0x7f54240ead00 2024-04-26T20:21:55.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f5432566640 1 -- 172.21.15.37:0/4230199400 >> v2:172.21.15.37:3300/0 conn(0x7f5424005e90 msgr2=0x7f54240062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f5432566640 1 --2- 172.21.15.37:0/4230199400 >> v2:172.21.15.37:3300/0 conn(0x7f5424005e90 0x7f54240062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f5432566640 1 -- 172.21.15.37:0/4230199400 >> v1:172.21.15.134:6789/0 conn(0x7f54240ead00 legacy=0x7f542404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f5432566640 1 -- 172.21.15.37:0/4230199400 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5424053ce0 con 0x7f5424006810 2024-04-26T20:21:55.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f5431564640 1 --2- 172.21.15.37:0/4230199400 >> v2:172.21.15.37:3300/0 conn(0x7f5424005e90 0x7f54240062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f694d593640 1 --2- 172.21.15.37:0/3874570685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69400ead00 0x7f694004fa90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6948051930 tx=0x7f69480a6f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bcd6656a4c08582 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7ff3f37fe640 1 -- 172.21.15.37:0/3659619276 >> v1:172.21.15.134:6789/0 conn(0x7ff3f406f170 legacy=0x7ff3f406f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7ff3f37fe640 1 -- 172.21.15.37:0/3659619276 >> v1:172.21.15.37:6789/0 conn(0x7ff3f415bdd0 legacy=0x7ff3f40d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7ff3f37fe640 1 -- 172.21.15.37:0/3659619276 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f40c7760 con 0x7ff3f4072fc0 2024-04-26T20:21:55.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.795+0000 7f47f9927640 1 -- start start 2024-04-26T20:21:55.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f693ffff640 1 -- 172.21.15.37:0/3874570685 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480b4070 con 0x7f69400ead00 2024-04-26T20:21:55.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f693ffff640 1 -- 172.21.15.37:0/3874570685 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69480a7d80 con 0x7f69400ead00 2024-04-26T20:21:55.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f693ffff640 1 -- 172.21.15.37:0/3874570685 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480af660 con 0x7f69400ead00 2024-04-26T20:21:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2646804821 0 0) 0x7f0e14003520 con 0x7f0e20054300 2024-04-26T20:21:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e200e30c0 msgr2=0x7f0e20057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e200e30c0 0x7f0e20057d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 >> v1:172.21.15.134:6789/0 conn(0x7f0e2003e840 legacy=0x7f0e20053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e20058b30 con 0x7f0e20054300 2024-04-26T20:21:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f0e28050f10 con 0x7f0e20054300 2024-04-26T20:21:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 >> v1:172.21.15.37:6789/0 conn(0x7f0e20054300 legacy=0x7f0e200068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1796747760 0 0) 0x7f4e5806e6c0 con 0x7f4e5814d2e0 2024-04-26T20:21:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e200e30c0 0x7f0e14003a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e14004070 0x7f0e14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e14004d40 0x7f0e140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20058850 con 0x7f0e14004070 2024-04-26T20:21:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0e140143f0 con 0x7f0e14004d40 2024-04-26T20:21:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0e140145f0 con 0x7f0e200e30c0 2024-04-26T20:21:55.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f778d658640 1 --2- 172.21.15.37:0/1339013501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f778806c240 0x7f778806c6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f77840097a0 tx=0x7f778402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d0e7902ae1d9e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 >> v2:172.21.15.134:3300/0 conn(0x7f4e58072780 msgr2=0x7f4e5806f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f4e56ffd640 1 --2- 172.21.15.37:0/807712707 >> v2:172.21.15.134:3300/0 conn(0x7f4e58072780 0x7f4e5806f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2ed54640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e14004d40 0x7f0e140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f778ce57640 1 -- 172.21.15.37:0/1339013501 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784037070 con 0x7f778806c240 2024-04-26T20:21:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2dd52640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e14004070 0x7f0e14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 >> v1:172.21.15.37:6789/0 conn(0x7f4e580cfa90 legacy=0x7f4e58072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f0e2e553640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e200e30c0 0x7f0e14003a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7fab24931640 1 --2- 172.21.15.37:0/1409042080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab2006c6e0 0x7fab20073e40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009870 tx=0x7fab1c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95052501356edb56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f778ce57640 1 -- 172.21.15.37:0/1339013501 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7784032440 con 0x7f778806c240 2024-04-26T20:21:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f0e2dd52640 1 -- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e200e30c0 msgr2=0x7f0e14003a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.796+0000 7f5432566640 1 --2- 172.21.15.37:0/4230199400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 0x7f5424006ca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f542c02e060 tx=0x7f542c0a6f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1736328972440f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f0e2dd52640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e200e30c0 0x7f0e14003a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f0e2dd52640 1 -- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e14004d40 msgr2=0x7f0e140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f0e2dd52640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e14004d40 0x7f0e140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f0e2dd52640 1 -- 172.21.15.37:0/359869543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e20058b30 con 0x7f0e14004070 2024-04-26T20:21:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f0e2ed54640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e14004d40 0x7f0e140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5806ef60 con 0x7f4e5814d2e0 2024-04-26T20:21:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f4e48001670 con 0x7f4e5814d2e0 2024-04-26T20:21:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7fab17fff640 1 -- 172.21.15.37:0/1409042080 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c037070 con 0x7fab2006c6e0 2024-04-26T20:21:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f778ce57640 1 -- 172.21.15.37:0/1339013501 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784037070 con 0x7f778806c240 2024-04-26T20:21:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f694e595640 1 -- 172.21.15.37:0/3874570685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69400ead00 msgr2=0x7f694004fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f694e595640 1 --2- 172.21.15.37:0/3874570685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69400ead00 0x7f694004fa90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6948051930 tx=0x7f69480a6f70 comp rx=0 tx=0).stop 2024-04-26T20:21:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f694e595640 1 -- 172.21.15.37:0/3874570685 shutdown_connections 2024-04-26T20:21:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f694e595640 1 --2- 172.21.15.37:0/3874570685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69400ead00 0x7f694004fa90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f694e595640 1 -- 172.21.15.37:0/3874570685 >> 172.21.15.37:0/3874570685 conn(0x7f6940009fb0 msgr2=0x7f694000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 >> v1:172.21.15.134:6789/0 conn(0x7f4e5814d2e0 legacy=0x7f4e580b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7fab17fff640 1 -- 172.21.15.37:0/1409042080 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab1c032440 con 0x7fab2006c6e0 2024-04-26T20:21:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f4e56ffd640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40003b40 0x7f4e40003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f4e56ffd640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e40004800 0x7f4e40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f5430d63640 1 -- 172.21.15.37:0/4230199400 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c0b4070 con 0x7f5424006810 2024-04-26T20:21:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f5430d63640 1 -- 172.21.15.37:0/4230199400 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f542c0a7d90 con 0x7f5424006810 2024-04-26T20:21:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f77900bd640 1 -- 172.21.15.37:0/1339013501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f778806c240 msgr2=0x7f778806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f77900bd640 1 --2- 172.21.15.37:0/1339013501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f778806c240 0x7f778806c6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f77840097a0 tx=0x7f778402a180 comp rx=0 tx=0).stop 2024-04-26T20:21:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f77900bd640 1 -- 172.21.15.37:0/1339013501 shutdown_connections 2024-04-26T20:21:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f77900bd640 1 --2- 172.21.15.37:0/1339013501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f778806c240 0x7f778806c6c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f77900bd640 1 --2- 172.21.15.37:0/1339013501 >> v2:172.21.15.37:3300/0 conn(0x7f77880cde50 0x7f77880cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f77900bd640 1 -- 172.21.15.37:0/1339013501 >> 172.21.15.37:0/1339013501 conn(0x7f77880d1e00 msgr2=0x7f77880d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f4e57fff640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40003b40 0x7f4e40003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f4e577fe640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e40004800 0x7f4e40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f0e2dd52640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e14004070 0x7f0e14004800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e18001720 tx=0x7f0e14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c1d40c118c74b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f4e56ffd640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e400051c0 0x7f4e4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f694e595640 1 -- 172.21.15.37:0/3874570685 shutdown_connections 2024-04-26T20:21:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5806ec80 con 0x7f4e40004800 2024-04-26T20:21:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4000c0f0 con 0x7f4e400051c0 2024-04-26T20:21:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f694e595640 1 -- 172.21.15.37:0/3874570685 wait complete. 2024-04-26T20:21:55.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4000c2f0 con 0x7f4e40003b40 2024-04-26T20:21:55.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f77900bd640 1 -- 172.21.15.37:0/1339013501 shutdown_connections 2024-04-26T20:21:55.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18001370 con 0x7f0e14004070 2024-04-26T20:21:55.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7ff3f37fe640 1 --2- 172.21.15.37:0/3659619276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f4072fc0 0x7ff3f4073440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40c7690 tx=0x7ff3ec02c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aff28f5c7b7c87e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e180052d0 con 0x7f0e14004070 2024-04-26T20:21:55.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f0e2d551640 1 -- 172.21.15.37:0/359869543 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18001370 con 0x7f0e14004070 2024-04-26T20:21:55.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7fab17fff640 1 -- 172.21.15.37:0/1409042080 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c032810 con 0x7fab2006c6e0 2024-04-26T20:21:55.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7fab26134640 1 -- 172.21.15.37:0/1409042080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab2006c6e0 msgr2=0x7fab20073e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7fab26134640 1 --2- 172.21.15.37:0/1409042080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab2006c6e0 0x7fab20073e40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009870 tx=0x7fab1c02a210 comp rx=0 tx=0).stop 2024-04-26T20:21:55.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f77900bd640 1 -- 172.21.15.37:0/1339013501 wait complete. 2024-04-26T20:21:55.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f5433fc9640 1 -- 172.21.15.37:0/4230199400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 msgr2=0x7f5424006ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f5433fc9640 1 --2- 172.21.15.37:0/4230199400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 0x7f5424006ca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f542c02e060 tx=0x7f542c0a6f40 comp rx=0 tx=0).stop 2024-04-26T20:21:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7ff3f2ffd640 1 -- 172.21.15.37:0/3659619276 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec007740 con 0x7ff3f4072fc0 2024-04-26T20:21:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7ff3f2ffd640 1 -- 172.21.15.37:0/3659619276 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ec0078e0 con 0x7ff3f4072fc0 2024-04-26T20:21:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7fab26134640 1 -- 172.21.15.37:0/1409042080 shutdown_connections 2024-04-26T20:21:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7fab26134640 1 --2- 172.21.15.37:0/1409042080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab2006c6e0 0x7fab20073e40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7fab26134640 1 --2- 172.21.15.37:0/1409042080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab20073a60 0x7fab2015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7fab26134640 1 -- 172.21.15.37:0/1409042080 >> 172.21.15.37:0/1409042080 conn(0x7fab200d13f0 msgr2=0x7fab200d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f4e5cbb4640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e400051c0 0x7f4e4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f5433fc9640 1 -- 172.21.15.37:0/4230199400 shutdown_connections 2024-04-26T20:21:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f5433fc9640 1 --2- 172.21.15.37:0/4230199400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 0x7f5424006ca0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f5433fc9640 1 --2- 172.21.15.37:0/4230199400 >> v2:172.21.15.37:3300/0 conn(0x7f5424005e90 0x7f54240062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f5433fc9640 1 -- 172.21.15.37:0/4230199400 >> 172.21.15.37:0/4230199400 conn(0x7f5424009fb0 msgr2=0x7f542400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7fab26134640 1 -- 172.21.15.37:0/1409042080 shutdown_connections 2024-04-26T20:21:55.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f4e57fff640 1 -- 172.21.15.37:0/807712707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e400051c0 msgr2=0x7f4e4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7fab26134640 1 -- 172.21.15.37:0/1409042080 wait complete. 2024-04-26T20:21:55.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f4e57fff640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e400051c0 0x7f4e4000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f4e57fff640 1 -- 172.21.15.37:0/807712707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e40004800 msgr2=0x7f4e40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f47f9927640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f47f40cfdd0 0x7f47f415f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f47f9927640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f40d1430 con 0x7f47f415fb10 2024-04-26T20:21:55.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f47f9927640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f40d0390 con 0x7f47f415f1f0 2024-04-26T20:21:55.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.797+0000 7f47f9927640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f47f40d0fa0 con 0x7f47f40cfdd0 2024-04-26T20:21:55.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f47f3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f47f40cfdd0 0x7f47f415f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f47f3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f47f40cfdd0 0x7f47f415f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35002/0 (socket says 172.21.15.37:35002) 2024-04-26T20:21:55.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f47f3fff640 1 -- 172.21.15.37:0/3199680492 learned_addr learned my addr 172.21.15.37:0/3199680492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f47f3fff640 1 -- 172.21.15.37:0/3199680492 >> v1:172.21.15.134:6789/0 conn(0x7f47f415f1f0 legacy=0x7f47f40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f47f3fff640 1 -- 172.21.15.37:0/3199680492 >> v1:172.21.15.37:6789/0 conn(0x7f47f415fb10 legacy=0x7f47f40d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.798+0000 7f47f3fff640 1 -- 172.21.15.37:0/3199680492 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f40d0670 con 0x7f47f40cfdd0 2024-04-26T20:21:55.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f0e2f555640 1 -- 172.21.15.37:0/359869543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e14004070 msgr2=0x7f0e14004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f0e2f555640 1 --2- 172.21.15.37:0/359869543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e14004070 0x7f0e14004800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e18001720 tx=0x7f0e14005830 comp rx=0 tx=0).stop 2024-04-26T20:21:55.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f0e2ed54640 1 -- 172.21.15.37:0/359869543 reap_dead start 2024-04-26T20:21:55.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f4e57fff640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e40004800 0x7f4e40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f4e57fff640 1 -- 172.21.15.37:0/807712707 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5806e6c0 con 0x7f4e40003b40 2024-04-26T20:21:55.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7ff3f90c2640 1 -- 172.21.15.37:0/3659619276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f4072fc0 msgr2=0x7ff3f4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7ff3f90c2640 1 --2- 172.21.15.37:0/3659619276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f4072fc0 0x7ff3f4073440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40c7690 tx=0x7ff3ec02c7e0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f4e5cbb4640 1 -- 172.21.15.37:0/807712707 reap_dead start 2024-04-26T20:21:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f0e2f555640 1 -- 172.21.15.37:0/359869543 shutdown_connections 2024-04-26T20:21:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f0e2f555640 1 -- 172.21.15.37:0/359869543 >> 172.21.15.37:0/359869543 conn(0x7f0e200559e0 msgr2=0x7f0e20055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f47f3fff640 1 --2- 172.21.15.37:0/3199680492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f40cfdd0 0x7f47f415f5d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009e50 tx=0x7f47ec02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668f4d84fa1928e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f0e2f555640 1 -- 172.21.15.37:0/359869543 shutdown_connections 2024-04-26T20:21:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.800+0000 7f0e2f555640 1 -- 172.21.15.37:0/359869543 wait complete. 2024-04-26T20:21:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.800+0000 7f77900bd640 1 Processor -- start 2024-04-26T20:21:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f4e577fe640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e40004800 0x7f4e40004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.800+0000 7f5433fc9640 1 -- 172.21.15.37:0/4230199400 shutdown_connections 2024-04-26T20:21:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.800+0000 7f47f37fe640 1 -- 172.21.15.37:0/3199680492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec036070 con 0x7f47f40cfdd0 2024-04-26T20:21:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.800+0000 7f47f37fe640 1 -- 172.21.15.37:0/3199680492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec031600 con 0x7f47f40cfdd0 2024-04-26T20:21:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.800+0000 7f47f37fe640 1 -- 172.21.15.37:0/3199680492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec02ae60 con 0x7f47f40cfdd0 2024-04-26T20:21:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.799+0000 7f694e595640 1 Processor -- start 2024-04-26T20:21:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.800+0000 7f694e595640 1 -- start start 2024-04-26T20:21:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.800+0000 7f5433fc9640 1 -- 172.21.15.37:0/4230199400 wait complete. 2024-04-26T20:21:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f4e57fff640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40003b40 0x7f4e40003f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e48007ed0 tx=0x7f4e40005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7faf83f2b9434876 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.800+0000 7f77900bd640 1 -- start start 2024-04-26T20:21:55.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f77900bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778806c240 0x7f7788151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f77900bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77880cde50 0x7f7788166230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f77900bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7788167350 0x7f7788166770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7fab26134640 1 Processor -- start 2024-04-26T20:21:55.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f47f9927640 1 -- 172.21.15.37:0/3199680492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f40cfdd0 msgr2=0x7f47f415f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f47f9927640 1 --2- 172.21.15.37:0/3199680492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f40cfdd0 0x7f47f415f5d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009e50 tx=0x7f47ec02a680 comp rx=0 tx=0).stop 2024-04-26T20:21:55.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f47f9927640 1 -- 172.21.15.37:0/3199680492 shutdown_connections 2024-04-26T20:21:55.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f47f9927640 1 --2- 172.21.15.37:0/3199680492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f40cfdd0 0x7f47f415f5d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f47f9927640 1 -- 172.21.15.37:0/3199680492 >> 172.21.15.37:0/3199680492 conn(0x7f47f40d4fe0 msgr2=0x7f47f40d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48036070 con 0x7f4e40003b40 2024-04-26T20:21:55.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f47f9927640 1 -- 172.21.15.37:0/3199680492 shutdown_connections 2024-04-26T20:21:55.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e48031620 con 0x7f4e40003b40 2024-04-26T20:21:55.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7fab26134640 1 -- start start 2024-04-26T20:21:55.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f694e595640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6940006ea0 0x7f694004c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f694e595640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694004ffd0 0x7f6940189570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f694e595640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69400ead00 0x7f69400e1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f47f9927640 1 -- 172.21.15.37:0/3199680492 wait complete. 2024-04-26T20:21:55.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f0e2f555640 1 Processor -- start 2024-04-26T20:21:55.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f694e595640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6940054940 con 0x7f694004ffd0 2024-04-26T20:21:55.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f694e595640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69400547c0 con 0x7f69400ead00 2024-04-26T20:21:55.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f694e595640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f694004f550 con 0x7f6940006ea0 2024-04-26T20:21:55.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f694d593640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6940006ea0 0x7f694004c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f694d593640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6940006ea0 0x7f694004c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:55624/0 (socket says 172.21.15.37:55624) 2024-04-26T20:21:55.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f694d593640 1 -- 172.21.15.37:0/3259060573 learned_addr learned my addr 172.21.15.37:0/3259060573 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f694cd92640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694004ffd0 0x7f6940189570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f0e2f555640 1 -- start start 2024-04-26T20:21:55.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f694dd94640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69400ead00 0x7f69400e1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7ff3f90c2640 1 -- 172.21.15.37:0/3659619276 shutdown_connections 2024-04-26T20:21:55.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7ff3f90c2640 1 --2- 172.21.15.37:0/3659619276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f4072fc0 0x7ff3f4073440 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7ff3f90c2640 1 -- 172.21.15.37:0/3659619276 >> 172.21.15.37:0/3659619276 conn(0x7ff3f406d350 msgr2=0x7ff3f406d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7ff3f90c2640 1 -- 172.21.15.37:0/3659619276 shutdown_connections 2024-04-26T20:21:55.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f4e56ffd640 1 -- 172.21.15.37:0/807712707 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48007af0 con 0x7f4e40003b40 2024-04-26T20:21:55.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f694cd92640 1 -- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6940006ea0 msgr2=0x7f694004c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f694cd92640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6940006ea0 0x7f694004c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f694cd92640 1 -- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69400ead00 msgr2=0x7f69400e1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f694cd92640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69400ead00 0x7f69400e1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f694cd92640 1 -- 172.21.15.37:0/3259060573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694004ce80 con 0x7f694004ffd0 2024-04-26T20:21:55.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f694cd92640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694004ffd0 0x7f6940189570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f693800c440 tx=0x7f6938001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f4e5e617640 1 -- 172.21.15.37:0/807712707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40003b40 msgr2=0x7f4e40003f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f4e5e617640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40003b40 0x7f4e40003f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e48007ed0 tx=0x7f4e40005830 comp rx=0 tx=0).stop 2024-04-26T20:21:55.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f5433fc9640 1 Processor -- start 2024-04-26T20:21:55.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f5433fc9640 1 -- start start 2024-04-26T20:21:55.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f77900bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7788073ce0 con 0x7f7788167350 2024-04-26T20:21:55.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f77900bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77880b15f0 con 0x7f778806c240 2024-04-26T20:21:55.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.801+0000 7f77900bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7788073fc0 con 0x7f77880cde50 2024-04-26T20:21:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f778de59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778806c240 0x7f7788151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f778de59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778806c240 0x7f7788151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:33618/0 (socket says 172.21.15.37:33618) 2024-04-26T20:21:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f778de59640 1 -- 172.21.15.37:0/3296271228 learned_addr learned my addr 172.21.15.37:0/3296271228 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f778d658640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77880cde50 0x7f7788166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f0e2f555640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e14004070 0x7f0e200c3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7fab26134640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab20073a60 0x7fab201f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f778d658640 1 -- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778806c240 msgr2=0x7f7788151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2f555640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e20006b70 0x7f0e20006ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2f555640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e200d27d0 0x7f0e200d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2f555640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0e200e34a0 con 0x7f0e14004070 2024-04-26T20:21:55.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2f555640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2003bfc0 con 0x7f0e200d27d0 2024-04-26T20:21:55.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2f555640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0e20057d90 con 0x7f0e20006b70 2024-04-26T20:21:55.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f4e5e617640 1 -- 172.21.15.37:0/807712707 shutdown_connections 2024-04-26T20:21:55.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f4e5e617640 1 --2- 172.21.15.37:0/807712707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40003b40 0x7f4e40003f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f4e5e617640 1 -- 172.21.15.37:0/807712707 >> 172.21.15.37:0/807712707 conn(0x7f4e580d0d50 msgr2=0x7f4e580d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:55.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f693dffb640 1 -- 172.21.15.37:0/3259060573 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938017070 con 0x7f694004ffd0 2024-04-26T20:21:55.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f693dffb640 1 -- 172.21.15.37:0/3259060573 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6938005ce0 con 0x7f694004ffd0 2024-04-26T20:21:55.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7f693dffb640 1 -- 172.21.15.37:0/3259060573 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380072f0 con 0x7f694004ffd0 2024-04-26T20:21:55.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f778d658640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778806c240 0x7f7788151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f778e65a640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7788167350 0x7f7788166770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f778d658640 1 -- 172.21.15.37:0/3296271228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7788167350 msgr2=0x7f7788166770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f778d658640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7788167350 0x7f7788166770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f778d658640 1 -- 172.21.15.37:0/3296271228 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77881528f0 con 0x7f77880cde50 2024-04-26T20:21:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f778de59640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778806c240 0x7f7788151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2dd52640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e20006b70 0x7f0e20006ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2dd52640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e20006b70 0x7f0e20006ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:55632/0 (socket says 172.21.15.37:55632) 2024-04-26T20:21:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2dd52640 1 -- 172.21.15.37:0/2643743589 learned_addr learned my addr 172.21.15.37:0/2643743589 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2e553640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e14004070 0x7f0e200c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2ed54640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e200d27d0 0x7f0e200d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f778d658640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77880cde50 0x7f7788166230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7784031000 tx=0x7f77840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7fab26134640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab200b1bb0 0x7fab200c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7fab26134640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab200c84c0 0x7fab201f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2dd52640 1 -- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e200d27d0 msgr2=0x7f0e200d2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2dd52640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e200d27d0 0x7f0e200d2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2dd52640 1 -- 172.21.15.37:0/2643743589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e14004070 msgr2=0x7f0e200c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2dd52640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e14004070 0x7f0e200c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f777e7fc640 1 -- 172.21.15.37:0/3296271228 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784037050 con 0x7f77880cde50 2024-04-26T20:21:55.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2dd52640 1 -- 172.21.15.37:0/2643743589 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e200d37a0 con 0x7f0e20006b70 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7fab24931640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab200b1bb0 0x7fab200c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7fab24931640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab200b1bb0 0x7fab200c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:55638/0 (socket says 172.21.15.37:55638) 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7fab24931640 1 -- 172.21.15.37:0/668141978 learned_addr learned my addr 172.21.15.37:0/668141978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab2015cb60 con 0x7fab200c84c0 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab2006d440 con 0x7fab20073a60 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7fab25933640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab200c84c0 0x7fab201f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fab2006cb20 con 0x7fab200b1bb0 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7fab25933640 1 -- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab200b1bb0 msgr2=0x7fab200c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7fab25933640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab200b1bb0 0x7fab200c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7fab25933640 1 -- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab20073a60 msgr2=0x7fab201f12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7fab25933640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab20073a60 0x7fab201f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7fab25933640 1 -- 172.21.15.37:0/668141978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab200c4c90 con 0x7fab200c84c0 2024-04-26T20:21:55.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f0e2dd52640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e20006b70 0x7f0e20006ff0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e18001840 tx=0x7f0e18001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f0e2e553640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e14004070 0x7f0e200c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:55.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f0e1effd640 1 -- 172.21.15.37:0/2643743589 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1803e020 con 0x7f0e20006b70 2024-04-26T20:21:55.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f0e1effd640 1 -- 172.21.15.37:0/2643743589 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18005480 con 0x7f0e20006b70 2024-04-26T20:21:55.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f0e1effd640 1 -- 172.21.15.37:0/2643743589 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18001370 con 0x7f0e20006b70 2024-04-26T20:21:55.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f0e2f555640 1 -- 172.21.15.37:0/2643743589 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e200d34c0 con 0x7f0e20006b70 2024-04-26T20:21:55.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f0e2f555640 1 -- 172.21.15.37:0/2643743589 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e200d93c0 con 0x7f0e20006b70 2024-04-26T20:21:55.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f694e595640 1 -- 172.21.15.37:0/3259060573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694004d160 con 0x7f694004ffd0 2024-04-26T20:21:55.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f694e595640 1 -- 172.21.15.37:0/3259060573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6940186f70 con 0x7f694004ffd0 2024-04-26T20:21:55.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.802+0000 7ff3f90c2640 1 -- 172.21.15.37:0/3659619276 wait complete. 2024-04-26T20:21:55.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7ff3f90c2640 1 Processor -- start 2024-04-26T20:21:55.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.803+0000 7f777e7fc640 1 -- 172.21.15.37:0/3296271228 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7784038040 con 0x7f77880cde50 2024-04-26T20:21:55.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f777e7fc640 1 -- 172.21.15.37:0/3296271228 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784006e10 con 0x7f77880cde50 2024-04-26T20:21:55.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f77900bd640 1 -- 172.21.15.37:0/3296271228 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7788152bd0 con 0x7f77880cde50 2024-04-26T20:21:55.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f77900bd640 1 -- 172.21.15.37:0/3296271228 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f77881535b0 con 0x7f77880cde50 2024-04-26T20:21:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f47f9927640 1 Processor -- start 2024-04-26T20:21:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f47f9927640 1 -- start start 2024-04-26T20:21:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f5433fc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424005e90 0x7f54240328b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f5433fc9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 0x7f5424035470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f5433fc9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54240ead00 0x7f542402df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f5433fc9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f54240547c0 con 0x7f5424005e90 2024-04-26T20:21:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f5433fc9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54240501a0 con 0x7f54240ead00 2024-04-26T20:21:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f5433fc9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5424054940 con 0x7f5424006810 2024-04-26T20:21:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f5431564640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 0x7f5424035470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f5431564640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 0x7f5424035470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:55646/0 (socket says 172.21.15.37:55646) 2024-04-26T20:21:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f5431564640 1 -- 172.21.15.37:0/2737134762 learned_addr learned my addr 172.21.15.37:0/2737134762 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7f5431d65640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424005e90 0x7f54240328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f5432566640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54240ead00 0x7f542402df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f4e5e617640 1 -- 172.21.15.37:0/807712707 shutdown_connections 2024-04-26T20:21:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f4e5e617640 1 -- 172.21.15.37:0/807712707 wait complete. 2024-04-26T20:21:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.804+0000 7ff3f90c2640 1 -- start start 2024-04-26T20:21:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7ff3f90c2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f406f170 0x7ff3f415a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7ff3f90c2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f4072fc0 0x7ff3f40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f4072fc0 0x7ff3f40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7ff3f90c2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f415bdd0 0x7ff3f40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7ff3f90c2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f40c7f60 con 0x7ff3f406f170 2024-04-26T20:21:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7ff3f90c2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f40c7ad0 con 0x7ff3f4072fc0 2024-04-26T20:21:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7ff3f90c2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f40c80e0 con 0x7ff3f415bdd0 2024-04-26T20:21:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f4072fc0 0x7ff3f40ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:33634/0 (socket says 172.21.15.37:33634) 2024-04-26T20:21:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7ff3f37fe640 1 -- 172.21.15.37:0/232554565 learned_addr learned my addr 172.21.15.37:0/232554565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f47f9927640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47f415f1f0 0x7f47f415c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f47f9927640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f415fb10 0x7f47f40ba460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f47f9927640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47f4161550 0x7f47f4161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f47f9927640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47f415ff20 con 0x7f47f415fb10 2024-04-26T20:21:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f47f9927640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47f40d0c30 con 0x7f47f4161550 2024-04-26T20:21:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f47f9927640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f47f41600a0 con 0x7f47f415f1f0 2024-04-26T20:21:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f77900bd640 1 -- 172.21.15.37:0/3296271228 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77880d2580 con 0x7f77880cde50 2024-04-26T20:21:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f5431564640 1 -- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54240ead00 msgr2=0x7f542402df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f5431564640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54240ead00 0x7f542402df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f5431564640 1 -- 172.21.15.37:0/2737134762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424005e90 msgr2=0x7f54240328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f5431564640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424005e90 0x7f54240328b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.805+0000 7f5431564640 1 -- 172.21.15.37:0/2737134762 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5424035cc0 con 0x7f5424006810 2024-04-26T20:21:55.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7fab25933640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab200c84c0 0x7fab201f5050 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c5a0 tx=0x7fab1000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7fab15ffb640 1 -- 172.21.15.37:0/668141978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10005280 con 0x7fab200c84c0 2024-04-26T20:21:55.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7fab15ffb640 1 -- 172.21.15.37:0/668141978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab10015020 con 0x7fab200c84c0 2024-04-26T20:21:55.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab200c4f70 con 0x7fab200c84c0 2024-04-26T20:21:55.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7fab15ffb640 1 -- 172.21.15.37:0/668141978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10003830 con 0x7fab200c84c0 2024-04-26T20:21:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7f0e2f555640 1 -- 172.21.15.37:0/2643743589 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e200057e0 con 0x7f0e20006b70 2024-04-26T20:21:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7f694e595640 1 -- 172.21.15.37:0/3259060573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6940005390 con 0x7f694004ffd0 2024-04-26T20:21:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7ff3f3fff640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f406f170 0x7ff3f415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7ff3f37fe640 1 -- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f415bdd0 msgr2=0x7ff3f40ceaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:21:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7ff3f37fe640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f415bdd0 0x7ff3f40ceaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7ff3f37fe640 1 -- 172.21.15.37:0/232554565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f406f170 msgr2=0x7ff3f415a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7ff3f37fe640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f406f170 0x7ff3f415a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7ff3f37fe640 1 -- 172.21.15.37:0/232554565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f40b4b40 con 0x7ff3f4072fc0 2024-04-26T20:21:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7f47f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f415fb10 0x7f47f40ba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7f47f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f415fb10 0x7f47f40ba460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35046/0 (socket says 172.21.15.37:35046) 2024-04-26T20:21:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7f47f3fff640 1 -- 172.21.15.37:0/1122638346 learned_addr learned my addr 172.21.15.37:0/1122638346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7f47f9126640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47f4161550 0x7f47f4161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7f47f8925640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47f415f1f0 0x7f47f415c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.806+0000 7f5431564640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 0x7f5424035470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f542800e980 tx=0x7f542800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab200c5530 con 0x7fab200c84c0 2024-04-26T20:21:55.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7f777e7fc640 1 -- 172.21.15.37:0/3296271228 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7784045020 con 0x7f77880cde50 2024-04-26T20:21:55.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7f47f3fff640 1 -- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47f415f1f0 msgr2=0x7f47f415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7f47f3fff640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47f415f1f0 0x7f47f415c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7f47f3fff640 1 -- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47f4161550 msgr2=0x7f47f4161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7f47f3fff640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47f4161550 0x7f47f4161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7f47f3fff640 1 -- 172.21.15.37:0/1122638346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f40b8d90 con 0x7f47f415fb10 2024-04-26T20:21:55.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7f0e1effd640 1 -- 172.21.15.37:0/2643743589 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e18041020 con 0x7f0e20006b70 2024-04-26T20:21:55.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7f4e5e617640 1 Processor -- start 2024-04-26T20:21:55.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7f4e5e617640 1 -- start start 2024-04-26T20:21:55.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7ff3f37fe640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f4072fc0 0x7ff3f40ce560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009d90 tx=0x7ff3ec004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7ff3f0ff9640 1 -- 172.21.15.37:0/232554565 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec041030 con 0x7ff3f4072fc0 2024-04-26T20:21:55.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7ff3f0ff9640 1 -- 172.21.15.37:0/232554565 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ec03d440 con 0x7ff3f4072fc0 2024-04-26T20:21:55.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7ff3f0ff9640 1 -- 172.21.15.37:0/232554565 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec03d7f0 con 0x7ff3f4072fc0 2024-04-26T20:21:55.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.807+0000 7f777e7fc640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f776c06c920 0x7f776c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7f777e7fc640 1 -- 172.21.15.37:0/3296271228 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f77840abf50 con 0x7f77880cde50 2024-04-26T20:21:55.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7f778de59640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f776c06c920 0x7f776c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7f693dffb640 1 -- 172.21.15.37:0/3259060573 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f693801a070 con 0x7f694004ffd0 2024-04-26T20:21:55.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7ff3f90c2640 1 -- 172.21.15.37:0/232554565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f40b4de0 con 0x7ff3f4072fc0 2024-04-26T20:21:55.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7f0e1effd640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0e0c06c8f0 0x7f0e0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7f0e1effd640 1 -- 172.21.15.37:0/2643743589 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0e180ab5a0 con 0x7f0e20006b70 2024-04-26T20:21:55.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7f4e5e617640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40003b40 0x7f4e58139e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7f4e5e617640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e581eea10 0x7f4e581eee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab20076890 con 0x7fab200c84c0 2024-04-26T20:21:55.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7f693dffb640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f692c06c9c0 0x7f692c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e5e617640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e580b58b0 0x7f4e580b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f693dffb640 1 -- 172.21.15.37:0/3259060573 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f693808aeb0 con 0x7f694004ffd0 2024-04-26T20:21:55.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f0e2e553640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0e0c06c8f0 0x7f0e0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7f47f3fff640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f415fb10 0x7f47f40ba460 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47ec02abf0 tx=0x7f47ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f694d593640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f692c06c9c0 0x7f692c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e581eea10 0x7f4e581eee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e5e617640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5806e3d0 con 0x7f4e580b58b0 2024-04-26T20:21:55.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e5e617640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5806e250 con 0x7f4e581eea10 2024-04-26T20:21:55.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e581eea10 0x7f4e581eee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:33648/0 (socket says 172.21.15.37:33648) 2024-04-26T20:21:55.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e5e617640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4e580cfee0 con 0x7f4e40003b40 2024-04-26T20:21:55.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e577fe640 1 -- 172.21.15.37:0/4084726350 learned_addr learned my addr 172.21.15.37:0/4084726350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:55.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.808+0000 7ff3f90c2640 1 -- 172.21.15.37:0/232554565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f40b53a0 con 0x7ff3f4072fc0 2024-04-26T20:21:55.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e57fff640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40003b40 0x7f4e58139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e5cbb4640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e580b58b0 0x7f4e580b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f694d593640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f692c06c9c0 0x7f692c06ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f694806aaf0 tx=0x7f69480ab000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f778de59640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f776c06c920 0x7f776c06ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f77880b2e40 tx=0x7f778000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f0e2e553640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0e0c06c8f0 0x7f0e0c06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0e24006fd0 tx=0x7f0e24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7fab15ffb640 1 -- 172.21.15.37:0/668141978 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab100039d0 con 0x7fab200c84c0 2024-04-26T20:21:55.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e577fe640 1 -- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40003b40 msgr2=0x7f4e58139e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e577fe640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40003b40 0x7f4e58139e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.809+0000 7f4e577fe640 1 -- 172.21.15.37:0/4084726350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e580b58b0 msgr2=0x7f4e580b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:55.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.810+0000 7f4e577fe640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e580b58b0 0x7f4e580b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.810+0000 7f4e577fe640 1 -- 172.21.15.37:0/4084726350 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e581ec420 con 0x7f4e581eea10 2024-04-26T20:21:55.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.810+0000 7f4e5cbb4640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e580b58b0 0x7f4e580b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:55.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.810+0000 7fab15ffb640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab0406c940 0x7fab0406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.810+0000 7f4e577fe640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e581eea10 0x7f4e581eee90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c001760 tx=0x7f4e4c0074f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.810+0000 7fab15ffb640 1 -- 172.21.15.37:0/668141978 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fab1008bb30 con 0x7fab200c84c0 2024-04-26T20:21:55.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.810+0000 7fab25132640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab0406c940 0x7fab0406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.810+0000 7ff3f90c2640 1 -- 172.21.15.37:0/232554565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f4076890 con 0x7ff3f4072fc0 2024-04-26T20:21:55.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.810+0000 7f4e54ff9640 1 -- 172.21.15.37:0/4084726350 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c007c20 con 0x7f4e581eea10 2024-04-26T20:21:55.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.810+0000 7f4e54ff9640 1 -- 172.21.15.37:0/4084726350 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e4c018070 con 0x7f4e581eea10 2024-04-26T20:21:55.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.810+0000 7f4e54ff9640 1 -- 172.21.15.37:0/4084726350 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00fb40 con 0x7f4e581eea10 2024-04-26T20:21:55.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.811+0000 7f4e5e617640 1 -- 172.21.15.37:0/4084726350 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e581ec700 con 0x7f4e581eea10 2024-04-26T20:21:55.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.811+0000 7f4e5e617640 1 -- 172.21.15.37:0/4084726350 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581eccc0 con 0x7f4e581eea10 2024-04-26T20:21:55.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.811+0000 7fab25132640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab0406c940 0x7fab0406ee00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fab18007e60 tx=0x7fab18007390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.811+0000 7f54227fc640 1 -- 172.21.15.37:0/2737134762 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428017070 con 0x7f5424006810 2024-04-26T20:21:55.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.811+0000 7f5433fc9640 1 -- 172.21.15.37:0/2737134762 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5424035fa0 con 0x7f5424006810 2024-04-26T20:21:55.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.811+0000 7f5433fc9640 1 -- 172.21.15.37:0/2737134762 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5424036560 con 0x7f5424006810 2024-04-26T20:21:55.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.811+0000 7f777e7fc640 1 -- 172.21.15.37:0/3296271228 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f77880d2580 con 0x7f77880cde50 2024-04-26T20:21:55.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.811+0000 7ff3f0ff9640 1 -- 172.21.15.37:0/232554565 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3ec03b040 con 0x7ff3f4072fc0 2024-04-26T20:21:55.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.812+0000 7ff3f0ff9640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff3e006c9c0 0x7ff3e006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.812+0000 7ff3f0ff9640 1 -- 172.21.15.37:0/232554565 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff3ec0ad090 con 0x7ff3f4072fc0 2024-04-26T20:21:55.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.812+0000 7f4e5e617640 1 -- 172.21.15.37:0/4084726350 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e58075c10 con 0x7f4e581eea10 2024-04-26T20:21:55.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.812+0000 7ff3f3fff640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff3e006c9c0 0x7ff3e006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.812+0000 7ff3f3fff640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff3e006c9c0 0x7ff3e006ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000f60 tx=0x7ff3e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.813+0000 7f5433fc9640 1 -- 172.21.15.37:0/2737134762 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5424005390 con 0x7f5424006810 2024-04-26T20:21:55.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.814+0000 7f47f17fa640 1 -- 172.21.15.37:0/1122638346 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec041070 con 0x7f47f415fb10 2024-04-26T20:21:55.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.814+0000 7f47f9927640 1 -- 172.21.15.37:0/1122638346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f40b9070 con 0x7f47f415fb10 2024-04-26T20:21:55.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.814+0000 7f47f9927640 1 -- 172.21.15.37:0/1122638346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f40b9630 con 0x7f47f415fb10 2024-04-26T20:21:55.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.814+0000 7f693dffb640 1 -- 172.21.15.37:0/3259060573 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6938058c70 con 0x7f694004ffd0 2024-04-26T20:21:55.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.816+0000 7f54227fc640 1 -- 172.21.15.37:0/2737134762 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5428009340 con 0x7f5424006810 2024-04-26T20:21:55.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.816+0000 7f54227fc640 1 -- 172.21.15.37:0/2737134762 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428021060 con 0x7f5424006810 2024-04-26T20:21:55.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.816+0000 7f0e1effd640 1 -- 172.21.15.37:0/2643743589 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e18048030 con 0x7f0e20006b70 2024-04-26T20:21:55.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.816+0000 7ff3f0ff9640 1 -- 172.21.15.37:0/232554565 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3ec07b650 con 0x7ff3f4072fc0 2024-04-26T20:21:55.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.816+0000 7f47f9927640 1 -- 172.21.15.37:0/1122638346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f40cf9b0 con 0x7f47f415fb10 2024-04-26T20:21:55.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.818+0000 7f4e54ff9640 1 -- 172.21.15.37:0/4084726350 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e4c00fce0 con 0x7f4e581eea10 2024-04-26T20:21:55.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.818+0000 7f54227fc640 1 -- 172.21.15.37:0/2737134762 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f542801a070 con 0x7f5424006810 2024-04-26T20:21:55.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.818+0000 7f4e54ff9640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4e4406c8f0 0x7f4e4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.818+0000 7f4e54ff9640 1 -- 172.21.15.37:0/4084726350 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4e4c08c380 con 0x7f4e581eea10 2024-04-26T20:21:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.818+0000 7f54227fc640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f541006c8f0 0x7f541006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.819+0000 7f54227fc640 1 -- 172.21.15.37:0/2737134762 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f542808bf70 con 0x7f5424006810 2024-04-26T20:21:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.819+0000 7f5431d65640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f541006c8f0 0x7f541006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.819+0000 7f4e57fff640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4e4406c8f0 0x7f4e4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.819+0000 7f5431d65640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f541006c8f0 0x7f541006edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f541c0011e0 tx=0x7f541c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.819+0000 7f4e57fff640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4e4406c8f0 0x7f4e4406edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e48033000 tx=0x7f4e48005040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:55.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.821+0000 7f54227fc640 1 -- 172.21.15.37:0/2737134762 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f542805a300 con 0x7f5424006810 2024-04-26T20:21:55.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.821+0000 7f4e54ff9640 1 -- 172.21.15.37:0/4084726350 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e4c05a140 con 0x7f4e581eea10 2024-04-26T20:21:55.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.824+0000 7f47f17fa640 1 -- 172.21.15.37:0/1122638346 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec037040 con 0x7f47f415fb10 2024-04-26T20:21:55.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.824+0000 7f47f17fa640 1 -- 172.21.15.37:0/1122638346 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec041070 con 0x7f47f415fb10 2024-04-26T20:21:55.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.825+0000 7fab15ffb640 1 -- 172.21.15.37:0/668141978 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab1001f020 con 0x7fab200c84c0 2024-04-26T20:21:55.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.826+0000 7f47f17fa640 1 -- 172.21.15.37:0/1122638346 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47ec007910 con 0x7f47f415fb10 2024-04-26T20:21:55.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.826+0000 7f47f17fa640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f47e006ccf0 0x7f47e006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:55.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.827+0000 7f47f17fa640 1 -- 172.21.15.37:0/1122638346 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f47ec031400 con 0x7f47f415fb10 2024-04-26T20:21:55.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.828+0000 7f47f8925640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f47e006ccf0 0x7f47e006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.832+0000 7f47f17fa640 1 -- 172.21.15.37:0/1122638346 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47f40cf9b0 con 0x7f47f415fb10 2024-04-26T20:21:55.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:55.844+0000 7f47f8925640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f47e006ccf0 0x7f47e006f1b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f47e8006fd0 tx=0x7f47e8009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:56.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.248+0000 7f694e595640 1 -- 172.21.15.37:0/3259060573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6940048e50 con 0x7f694004ffd0 2024-04-26T20:21:56.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.249+0000 7f693dffb640 1 -- 172.21.15.37:0/3259060573 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f693805cf70 con 0x7f694004ffd0 2024-04-26T20:21:56.250 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.252+0000 7f692b7fe640 1 -- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f692c06c9c0 msgr2=0x7f692c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.252+0000 7f692b7fe640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f692c06c9c0 0x7f692c06ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f694806aaf0 tx=0x7f69480ab000 comp rx=0 tx=0).stop 2024-04-26T20:21:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.252+0000 7f692b7fe640 1 -- 172.21.15.37:0/3259060573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694004ffd0 msgr2=0x7f6940189570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.252+0000 7f692b7fe640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694004ffd0 0x7f6940189570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f693800c440 tx=0x7f6938001e70 comp rx=0 tx=0).stop 2024-04-26T20:21:56.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.253+0000 7f692b7fe640 1 -- 172.21.15.37:0/3259060573 shutdown_connections 2024-04-26T20:21:56.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.253+0000 7f692b7fe640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69400ead00 0x7f69400e1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.253+0000 7ff3f90c2640 1 -- 172.21.15.37:0/232554565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff3f40c70c0 con 0x7ff3f4072fc0 2024-04-26T20:21:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.253+0000 7f692b7fe640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694004ffd0 0x7f6940189570 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.253+0000 7f692b7fe640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f692c06c9c0 0x7f692c06ee80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.253+0000 7f692b7fe640 1 --2- 172.21.15.37:0/3259060573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6940006ea0 0x7f694004c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.253+0000 7f692b7fe640 1 -- 172.21.15.37:0/3259060573 >> 172.21.15.37:0/3259060573 conn(0x7f6940009fb0 msgr2=0x7f6940036ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.253+0000 7f77900bd640 1 -- 172.21.15.37:0/3296271228 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7788076900 con 0x7f77880cde50 2024-04-26T20:21:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.254+0000 7f692b7fe640 1 -- 172.21.15.37:0/3259060573 shutdown_connections 2024-04-26T20:21:56.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.253+0000 7ff3f0ff9640 1 -- 172.21.15.37:0/232554565 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff3ec07af50 con 0x7ff3f4072fc0 2024-04-26T20:21:56.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.254+0000 7f692b7fe640 1 -- 172.21.15.37:0/3259060573 wait complete. 2024-04-26T20:21:56.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.254+0000 7f777e7fc640 1 -- 172.21.15.37:0/3296271228 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7784080030 con 0x7f77880cde50 2024-04-26T20:21:56.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.255+0000 7f0e2f555640 1 -- 172.21.15.37:0/2643743589 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0e20039a70 con 0x7f0e20006b70 2024-04-26T20:21:56.257 INFO:teuthology.orchestra.run.smithi037.stdout:55834574854 2024-04-26T20:21:56.257 INFO:teuthology.orchestra.run.smithi037.stdout:55834574854 2024-04-26T20:21:56.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.255+0000 7f0e1effd640 1 -- 172.21.15.37:0/2643743589 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e180799c0 con 0x7f0e20006b70 2024-04-26T20:21:56.258 INFO:teuthology.orchestra.run.smithi037.stdout:55834574854 2024-04-26T20:21:56.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7ff3f90c2640 1 -- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff3e006c9c0 msgr2=0x7ff3e006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7ff3f90c2640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff3e006c9c0 0x7ff3e006ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000f60 tx=0x7ff3e4009040 comp rx=0 tx=0).stop 2024-04-26T20:21:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7ff3f90c2640 1 -- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f4072fc0 msgr2=0x7ff3f40ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7ff3f90c2640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f4072fc0 0x7ff3f40ce560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009d90 tx=0x7ff3ec004a40 comp rx=0 tx=0).stop 2024-04-26T20:21:56.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7f77900bd640 1 -- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f776c06c920 msgr2=0x7f776c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7f77900bd640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f776c06c920 0x7f776c06ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f77880b2e40 tx=0x7f778000b040 comp rx=0 tx=0).stop 2024-04-26T20:21:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7f77900bd640 1 -- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77880cde50 msgr2=0x7f7788166230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7f77900bd640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77880cde50 0x7f7788166230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7784031000 tx=0x7f77840046c0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7f77900bd640 1 -- 172.21.15.37:0/3296271228 shutdown_connections 2024-04-26T20:21:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7f77900bd640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7788167350 0x7f7788166770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7f77900bd640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f776c06c920 0x7f776c06ede0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7f77900bd640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77880cde50 0x7f7788166230 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7f77900bd640 1 --2- 172.21.15.37:0/3296271228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778806c240 0x7f7788151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.257+0000 7f77900bd640 1 -- 172.21.15.37:0/3296271228 >> 172.21.15.37:0/3296271228 conn(0x7f77880d1e00 msgr2=0x7f77880c4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.258+0000 7ff3f90c2640 1 -- 172.21.15.37:0/232554565 shutdown_connections 2024-04-26T20:21:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.258+0000 7ff3f90c2640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff3e006c9c0 0x7ff3e006ee80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.258+0000 7ff3f90c2640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f415bdd0 0x7ff3f40ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.258+0000 7ff3f90c2640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f4072fc0 0x7ff3f40ce560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.258+0000 7ff3f90c2640 1 --2- 172.21.15.37:0/232554565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f406f170 0x7ff3f415a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.258+0000 7ff3f90c2640 1 -- 172.21.15.37:0/232554565 >> 172.21.15.37:0/232554565 conn(0x7ff3f406d350 msgr2=0x7ff3f415d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.258+0000 7f77900bd640 1 -- 172.21.15.37:0/3296271228 shutdown_connections 2024-04-26T20:21:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.258+0000 7f77900bd640 1 -- 172.21.15.37:0/3296271228 wait complete. 2024-04-26T20:21:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.258+0000 7ff3f90c2640 1 -- 172.21.15.37:0/232554565 shutdown_connections 2024-04-26T20:21:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.258+0000 7ff3f90c2640 1 -- 172.21.15.37:0/232554565 wait complete. 2024-04-26T20:21:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.259+0000 7f0e2f555640 1 -- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0e0c06c8f0 msgr2=0x7f0e0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.259+0000 7f0e2f555640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0e0c06c8f0 0x7f0e0c06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0e24006fd0 tx=0x7f0e24008040 comp rx=0 tx=0).stop 2024-04-26T20:21:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.259+0000 7f0e2f555640 1 -- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e20006b70 msgr2=0x7f0e20006ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.259+0000 7f0e2f555640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e20006b70 0x7f0e20006ff0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e18001840 tx=0x7f0e18001120 comp rx=0 tx=0).stop 2024-04-26T20:21:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.259+0000 7f0e2f555640 1 -- 172.21.15.37:0/2643743589 shutdown_connections 2024-04-26T20:21:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.259+0000 7f0e2f555640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e200d27d0 0x7f0e200d2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.259+0000 7f0e2f555640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0e0c06c8f0 0x7f0e0c06edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.259+0000 7f0e2f555640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e20006b70 0x7f0e20006ff0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.259+0000 7f0e2f555640 1 --2- 172.21.15.37:0/2643743589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e14004070 0x7f0e200c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.260+0000 7f0e2f555640 1 -- 172.21.15.37:0/2643743589 >> 172.21.15.37:0/2643743589 conn(0x7f0e200559e0 msgr2=0x7f0e2003c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.260+0000 7f0e2f555640 1 -- 172.21.15.37:0/2643743589 shutdown_connections 2024-04-26T20:21:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.260+0000 7f0e2f555640 1 -- 172.21.15.37:0/2643743589 wait complete. 2024-04-26T20:21:56.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.262+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fab20073280 con 0x7fab200c84c0 2024-04-26T20:21:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.263+0000 7f5433fc9640 1 -- 172.21.15.37:0/2737134762 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f54240489c0 con 0x7f5424006810 2024-04-26T20:21:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.263+0000 7fab15ffb640 1 -- 172.21.15.37:0/668141978 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fab1005a040 con 0x7fab200c84c0 2024-04-26T20:21:56.265 INFO:teuthology.orchestra.run.smithi037.stdout:55834574854 2024-04-26T20:21:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.264+0000 7f54227fc640 1 -- 172.21.15.37:0/2737134762 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5428059c00 con 0x7f5424006810 2024-04-26T20:21:56.266 INFO:teuthology.orchestra.run.smithi037.stdout:55834574853 2024-04-26T20:21:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.267+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab0406c940 msgr2=0x7fab0406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.267+0000 7fab26134640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab0406c940 0x7fab0406ee00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fab18007e60 tx=0x7fab18007390 comp rx=0 tx=0).stop 2024-04-26T20:21:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.267+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab200c84c0 msgr2=0x7fab201f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.267+0000 7f540ffff640 1 -- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f541006c8f0 msgr2=0x7f541006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f540ffff640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f541006c8f0 0x7f541006edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f541c0011e0 tx=0x7f541c009040 comp rx=0 tx=0).stop 2024-04-26T20:21:56.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.267+0000 7fab26134640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab200c84c0 0x7fab201f5050 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c5a0 tx=0x7fab1000ca60 comp rx=0 tx=0).stop 2024-04-26T20:21:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f540ffff640 1 -- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 msgr2=0x7f5424035470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f540ffff640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 0x7f5424035470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f542800e980 tx=0x7f542800ee40 comp rx=0 tx=0).stop 2024-04-26T20:21:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f540ffff640 1 -- 172.21.15.37:0/2737134762 shutdown_connections 2024-04-26T20:21:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f540ffff640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54240ead00 0x7f542402df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f540ffff640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f541006c8f0 0x7f541006edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f540ffff640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5424006810 0x7f5424035470 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f540ffff640 1 --2- 172.21.15.37:0/2737134762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424005e90 0x7f54240328b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f540ffff640 1 -- 172.21.15.37:0/2737134762 >> 172.21.15.37:0/2737134762 conn(0x7f5424009fb0 msgr2=0x7f5424006bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f47f9927640 1 -- 172.21.15.37:0/1122638346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f47f4076890 con 0x7f47f415fb10 2024-04-26T20:21:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f540ffff640 1 -- 172.21.15.37:0/2737134762 shutdown_connections 2024-04-26T20:21:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7f540ffff640 1 -- 172.21.15.37:0/2737134762 wait complete. 2024-04-26T20:21:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 shutdown_connections 2024-04-26T20:21:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7fab26134640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab200c84c0 0x7fab201f5050 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7fab26134640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab0406c940 0x7fab0406ee00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7fab26134640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab200b1bb0 0x7fab200c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7fab26134640 1 --2- 172.21.15.37:0/668141978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab20073a60 0x7fab201f12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.268+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 >> 172.21.15.37:0/668141978 conn(0x7fab200d13f0 msgr2=0x7fab200cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.269+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 shutdown_connections 2024-04-26T20:21:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.269+0000 7fab26134640 1 -- 172.21.15.37:0/668141978 wait complete. 2024-04-26T20:21:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.269+0000 7f47f17fa640 1 -- 172.21.15.37:0/1122638346 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f47f40cf9b0 con 0x7f47f415fb10 2024-04-26T20:21:56.273 INFO:teuthology.orchestra.run.smithi037.stdout:55834574854 2024-04-26T20:21:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.269+0000 7f4e5e617640 1 -- 172.21.15.37:0/4084726350 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4e581ecec0 con 0x7f4e581eea10 2024-04-26T20:21:56.273 INFO:teuthology.orchestra.run.smithi037.stdout:55834574854 2024-04-26T20:21:56.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.269+0000 7f4e54ff9640 1 -- 172.21.15.37:0/4084726350 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4e581ecec0 con 0x7f4e581eea10 2024-04-26T20:21:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.273+0000 7f47f9927640 1 -- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f47e006ccf0 msgr2=0x7f47e006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.273+0000 7f47f9927640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f47e006ccf0 0x7f47e006f1b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f47e8006fd0 tx=0x7f47e8009060 comp rx=0 tx=0).stop 2024-04-26T20:21:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.273+0000 7f47f9927640 1 -- 172.21.15.37:0/1122638346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f415fb10 msgr2=0x7f47f40ba460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.273+0000 7f47f9927640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f415fb10 0x7f47f40ba460 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47ec02abf0 tx=0x7f47ec004a40 comp rx=0 tx=0).stop 2024-04-26T20:21:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.273+0000 7f47f9927640 1 -- 172.21.15.37:0/1122638346 shutdown_connections 2024-04-26T20:21:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.273+0000 7f47f9927640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47f4161550 0x7f47f4161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.273+0000 7f47f9927640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f415fb10 0x7f47f40ba460 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.273+0000 7f47f9927640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f47e006ccf0 0x7f47e006f1b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.273+0000 7f47f9927640 1 --2- 172.21.15.37:0/1122638346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47f415f1f0 0x7f47f415c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.273+0000 7f47f9927640 1 -- 172.21.15.37:0/1122638346 >> 172.21.15.37:0/1122638346 conn(0x7f47f40d4fe0 msgr2=0x7f47f40d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.274+0000 7f47f9927640 1 -- 172.21.15.37:0/1122638346 shutdown_connections 2024-04-26T20:21:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.274+0000 7f47f9927640 1 -- 172.21.15.37:0/1122638346 wait complete. 2024-04-26T20:21:56.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.274+0000 7f4e5e617640 1 -- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4e4406c8f0 msgr2=0x7f4e4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.274+0000 7f4e5e617640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4e4406c8f0 0x7f4e4406edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e48033000 tx=0x7f4e48005040 comp rx=0 tx=0).stop 2024-04-26T20:21:56.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.274+0000 7f4e5e617640 1 -- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e581eea10 msgr2=0x7f4e581eee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.274+0000 7f4e5e617640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e581eea10 0x7f4e581eee90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c001760 tx=0x7f4e4c0074f0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.275+0000 7f4e5e617640 1 -- 172.21.15.37:0/4084726350 shutdown_connections 2024-04-26T20:21:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.275+0000 7f4e5e617640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e580b58b0 0x7f4e580b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.275+0000 7f4e5e617640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e581eea10 0x7f4e581eee90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.275+0000 7f4e5e617640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4e4406c8f0 0x7f4e4406edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.275+0000 7f4e5e617640 1 --2- 172.21.15.37:0/4084726350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40003b40 0x7f4e58139e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.275+0000 7f4e5e617640 1 -- 172.21.15.37:0/4084726350 >> 172.21.15.37:0/4084726350 conn(0x7f4e580d0d50 msgr2=0x7f4e5814b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:56.279 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-04-26T20:21:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.276+0000 7f4e5e617640 1 -- 172.21.15.37:0/4084726350 shutdown_connections 2024-04-26T20:21:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:56.276+0000 7f4e5e617640 1 -- 172.21.15.37:0/4084726350 wait complete. 2024-04-26T20:21:56.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-04-26T20:21:56.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-04-26T20:21:56.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-04-26T20:21:56.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-04-26T20:21:56.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-04-26T20:21:56.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-04-26T20:21:56.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-04-26T20:21:57.280 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T20:21:57.284 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T20:21:57.285 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T20:21:57.285 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T20:21:57.290 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T20:21:57.293 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T20:21:57.293 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T20:21:57.295 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T20:21:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.448+0000 7efd9aac2640 1 Processor -- start 2024-04-26T20:21:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.449+0000 7efd9aac2640 1 -- start start 2024-04-26T20:21:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.450+0000 7efd9aac2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efd8c0e00a0 0x7efd8c034fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.450+0000 7efd9aac2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c050430 0x7efd8c038de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.450+0000 7efd9aac2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd8c034d10 con 0x7efd8c039320 2024-04-26T20:21:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.450+0000 7efd9aac2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7efd8c050930 con 0x7efd8c0e00a0 2024-04-26T20:21:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.450+0000 7efd9aac2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c007d30 con 0x7efd8c050430 2024-04-26T20:21:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.451+0000 7efd9a2c1640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7efd8c039320 0x7efd8c008af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:60806/0 (socket says 172.21.15.37:60806) 2024-04-26T20:21:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.451+0000 7efd9a2c1640 1 -- 172.21.15.37:0/789332686 learned_addr learned my addr 172.21.15.37:0/789332686 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:57.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.451+0000 7efd992bf640 1 --2- 172.21.15.37:0/789332686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c050430 0x7efd8c038de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.452+0000 7efd99ac0640 1 --2- 172.21.15.37:0/789332686 >> v2:172.21.15.134:3300/0 conn(0x7efd8c0e00a0 0x7efd8c034fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.452+0000 7efd992bf640 1 -- 172.21.15.37:0/789332686 >> v2:172.21.15.134:3300/0 conn(0x7efd8c0e00a0 msgr2=0x7efd8c034fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.452+0000 7efd992bf640 1 --2- 172.21.15.37:0/789332686 >> v2:172.21.15.134:3300/0 conn(0x7efd8c0e00a0 0x7efd8c034fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.452+0000 7efd992bf640 1 -- 172.21.15.37:0/789332686 >> v1:172.21.15.134:6789/0 conn(0x7efd8c039320 legacy=0x7efd8c008af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.452+0000 7efd992bf640 1 -- 172.21.15.37:0/789332686 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c008010 con 0x7efd8c050430 2024-04-26T20:21:57.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.454+0000 7efd992bf640 1 --2- 172.21.15.37:0/789332686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c050430 0x7efd8c038de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd900097a0 tx=0x7efd9002c210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70307e423ea20066 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.454+0000 7efd98abe640 1 -- 172.21.15.37:0/789332686 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90039070 con 0x7efd8c050430 2024-04-26T20:21:57.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.454+0000 7efd98abe640 1 -- 172.21.15.37:0/789332686 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd90034440 con 0x7efd8c050430 2024-04-26T20:21:57.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.455+0000 7efd9aac2640 1 -- 172.21.15.37:0/789332686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c050430 msgr2=0x7efd8c038de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.455+0000 7efd9aac2640 1 --2- 172.21.15.37:0/789332686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c050430 0x7efd8c038de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd900097a0 tx=0x7efd9002c210 comp rx=0 tx=0).stop 2024-04-26T20:21:57.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.456+0000 7efd9aac2640 1 -- 172.21.15.37:0/789332686 shutdown_connections 2024-04-26T20:21:57.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.456+0000 7efd9aac2640 1 --2- 172.21.15.37:0/789332686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c050430 0x7efd8c038de0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.456+0000 7efd9aac2640 1 --2- 172.21.15.37:0/789332686 >> v2:172.21.15.134:3300/0 conn(0x7efd8c0e00a0 0x7efd8c034fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.456+0000 7efd9aac2640 1 -- 172.21.15.37:0/789332686 >> 172.21.15.37:0/789332686 conn(0x7efd8c055bd0 msgr2=0x7efd8c055fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.456+0000 7efd9aac2640 1 -- 172.21.15.37:0/789332686 shutdown_connections 2024-04-26T20:21:57.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.456+0000 7efd9aac2640 1 -- 172.21.15.37:0/789332686 wait complete. 2024-04-26T20:21:57.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.458+0000 7efd9aac2640 1 Processor -- start 2024-04-26T20:21:57.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.458+0000 7efd9aac2640 1 -- start start 2024-04-26T20:21:57.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.460+0000 7efd9aac2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c039320 0x7efd8c17a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.460+0000 7efd9aac2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c0e00a0 0x7efd8c188ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.460+0000 7efd99ac0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c039320 0x7efd8c17a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.460+0000 7efd99ac0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c039320 0x7efd8c17a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:55674/0 (socket says 172.21.15.37:55674) 2024-04-26T20:21:57.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.460+0000 7efd992bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c0e00a0 0x7efd8c188ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.461+0000 7efd992bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c0e00a0 0x7efd8c188ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35076/0 (socket says 172.21.15.37:35076) 2024-04-26T20:21:57.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.461+0000 7efd99ac0640 1 -- 172.21.15.37:0/1837195955 learned_addr learned my addr 172.21.15.37:0/1837195955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:57.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.461+0000 7efd9aac2640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd8c0e1fd0 0x7efd8c0e2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.461+0000 7efd9aac2640 1 -- 172.21.15.37:0/1837195955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c009200 con 0x7efd8c0e00a0 2024-04-26T20:21:57.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.461+0000 7efd9aac2640 1 -- 172.21.15.37:0/1837195955 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c0085d0 con 0x7efd8c0e1fd0 2024-04-26T20:21:57.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.461+0000 7efd9aac2640 1 -- 172.21.15.37:0/1837195955 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c009380 con 0x7efd8c039320 2024-04-26T20:21:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.461+0000 7efd9a2c1640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd8c0e1fd0 0x7efd8c0e2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.462+0000 7efd992bf640 1 -- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c039320 msgr2=0x7efd8c17a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.462+0000 7efd992bf640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c039320 0x7efd8c17a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.462+0000 7efd992bf640 1 -- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd8c0e1fd0 msgr2=0x7efd8c0e2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.462+0000 7efd992bf640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd8c0e1fd0 0x7efd8c0e2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.462+0000 7efd992bf640 1 -- 172.21.15.37:0/1837195955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c17ac30 con 0x7efd8c0e00a0 2024-04-26T20:21:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.463+0000 7efd992bf640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c0e00a0 0x7efd8c188ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efd8c050ab0 tx=0x7efd900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.463+0000 7efd8a7fc640 1 -- 172.21.15.37:0/1837195955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90039040 con 0x7efd8c0e00a0 2024-04-26T20:21:57.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.464+0000 7efd9aac2640 1 -- 172.21.15.37:0/1837195955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c187c40 con 0x7efd8c0e00a0 2024-04-26T20:21:57.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.464+0000 7efd9aac2640 1 -- 172.21.15.37:0/1837195955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c188200 con 0x7efd8c0e00a0 2024-04-26T20:21:57.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.467+0000 7efd8a7fc640 1 -- 172.21.15.37:0/1837195955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd90034440 con 0x7efd8c0e00a0 2024-04-26T20:21:57.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.467+0000 7efd8a7fc640 1 -- 172.21.15.37:0/1837195955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90003a30 con 0x7efd8c0e00a0 2024-04-26T20:21:57.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.469+0000 7efd8a7fc640 1 -- 172.21.15.37:0/1837195955 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd90003c90 con 0x7efd8c0e00a0 2024-04-26T20:21:57.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.470+0000 7efd8a7fc640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efd7806c9c0 0x7efd7806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.470+0000 7efd8a7fc640 1 -- 172.21.15.37:0/1837195955 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efd900ac0e0 con 0x7efd8c0e00a0 2024-04-26T20:21:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.472+0000 7efd99ac0640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efd7806c9c0 0x7efd7806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.472+0000 7efd9aac2640 1 -- 172.21.15.37:0/1837195955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c005290 con 0x7efd8c0e00a0 2024-04-26T20:21:57.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.482+0000 7efd8a7fc640 1 -- 172.21.15.37:0/1837195955 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd9007a610 con 0x7efd8c0e00a0 2024-04-26T20:21:57.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.484+0000 7efd99ac0640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efd7806c9c0 0x7efd7806ee80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd84006fd0 tx=0x7efd84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.506+0000 7fe9d5e56640 1 Processor -- start 2024-04-26T20:21:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.506+0000 7fe9d5e56640 1 -- start start 2024-04-26T20:21:57.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.507+0000 7fe9d5e56640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe9c80507a0 0x7fe9c8054cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.507+0000 7ffb8eb82640 1 Processor -- start 2024-04-26T20:21:57.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.507+0000 7fe9d5e56640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9c8053c30 con 0x7fe9c803ad50 2024-04-26T20:21:57.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.507+0000 7fe9d5e56640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9c8053f10 con 0x7fe9c80df6f0 2024-04-26T20:21:57.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.507+0000 7fe9d5e56640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe9c80541f0 con 0x7fe9c80507a0 2024-04-26T20:21:57.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7ffb8eb82640 1 -- start start 2024-04-26T20:21:57.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7f062d031640 1 Processor -- start 2024-04-26T20:21:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7f062d031640 1 -- start start 2024-04-26T20:21:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7fe9cffff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe9c80df6f0 0x7fe9c8050090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:60818/0 (socket says 172.21.15.37:60818) 2024-04-26T20:21:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7fe9cffff640 1 -- 172.21.15.37:0/2287380832 learned_addr learned my addr 172.21.15.37:0/2287380832 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7fe9d5655640 1 --2- 172.21.15.37:0/2287380832 >> v2:172.21.15.134:3300/0 conn(0x7fe9c80507a0 0x7fe9c8054cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7f413d395640 1 Processor -- start 2024-04-26T20:21:57.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 870240651 0 0) 0x7fe9c8053f10 con 0x7fe9c80df6f0 2024-04-26T20:21:57.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9c0003520 con 0x7fe9c80df6f0 2024-04-26T20:21:57.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7f413d395640 1 -- start start 2024-04-26T20:21:57.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1230417661 0 0) 0x7fe9c0003520 con 0x7fe9c80df6f0 2024-04-26T20:21:57.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.508+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 >> v2:172.21.15.134:3300/0 conn(0x7fe9c80507a0 msgr2=0x7fe9c8054cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9cf7fe640 1 --2- 172.21.15.37:0/2287380832 >> v2:172.21.15.134:3300/0 conn(0x7fe9c80507a0 0x7fe9c8054cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 >> v1:172.21.15.37:6789/0 conn(0x7fe9c803ad50 legacy=0x7fe9c8053400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c80544d0 con 0x7fe9c80df6f0 2024-04-26T20:21:57.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7feb8f59e640 1 Processor -- start 2024-04-26T20:21:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fe9d0076820 con 0x7fe9c80df6f0 2024-04-26T20:21:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7feb8f59e640 1 -- start start 2024-04-26T20:21:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7ffb8eb82640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffb88159a70 0x7ffb880b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 >> v1:172.21.15.134:6789/0 conn(0x7fe9c80df6f0 legacy=0x7fe9c8050090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7ffb8eb82640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb880cd7b0 0x7ffb880cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9cf7fe640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 0x7fe9c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7ffb8eb82640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8815fbe0 con 0x7ffb880afae0 2024-04-26T20:21:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7ffb8eb82640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ffb881607f0 con 0x7ffb88159a70 2024-04-26T20:21:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7ffb8eb82640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8815fec0 con 0x7ffb880cd7b0 2024-04-26T20:21:57.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7f062d031640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 0x7f06200062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7f062d031640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0620006830 0x7f0620006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7f062d031640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0620048420 con 0x7f0620048040 2024-04-26T20:21:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7f062d031640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f06200501a0 con 0x7f0620006830 2024-04-26T20:21:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7f062d031640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f06200547a0 con 0x7f0620005e90 2024-04-26T20:21:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7f062c830640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0620006830 0x7f0620006cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7f062c830640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0620006830 0x7f0620006cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35080/0 (socket says 172.21.15.37:35080) 2024-04-26T20:21:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7f062c830640 1 -- 172.21.15.37:0/1406121097 learned_addr learned my addr 172.21.15.37:0/1406121097 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7f06277fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 0x7f06200062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7ffb8e381640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ffb880afae0 0x7ffb881630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:60826/0 (socket says 172.21.15.37:60826) 2024-04-26T20:21:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7ffb8e381640 1 -- 172.21.15.37:0/1459199074 learned_addr learned my addr 172.21.15.37:0/1459199074 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:57.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7ffb8db80640 1 --2- 172.21.15.37:0/1459199074 >> v2:172.21.15.134:3300/0 conn(0x7ffb88159a70 0x7ffb880b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7ffb8d37f640 1 --2- 172.21.15.37:0/1459199074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb880cd7b0 0x7ffb880cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9cf7fe640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9c0004310 0x7fe9c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9cf7fe640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c0004fe0 0x7fe9c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9d4e54640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 0x7fe9c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c80541f0 con 0x7fe9c0004fe0 2024-04-26T20:21:57.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c000bf30 con 0x7fe9c00039b0 2024-04-26T20:21:57.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c000c130 con 0x7fe9c0004310 2024-04-26T20:21:57.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7fe9cffff640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9c0004310 0x7fe9c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7fe9d5655640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c0004fe0 0x7fe9c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7ffb8cb7e640 1 -- 172.21.15.37:0/1459199074 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4050401209 0 0) 0x7ffb8815fbe0 con 0x7ffb880afae0 2024-04-26T20:21:57.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7ffb8cb7e640 1 -- 172.21.15.37:0/1459199074 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb74003520 con 0x7ffb880afae0 2024-04-26T20:21:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7f06277fe640 1 -- 172.21.15.37:0/1406121097 >> v2:172.21.15.37:3300/0 conn(0x7f0620006830 msgr2=0x7f0620006cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7f06277fe640 1 --2- 172.21.15.37:0/1406121097 >> v2:172.21.15.37:3300/0 conn(0x7f0620006830 0x7f0620006cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7f06277fe640 1 -- 172.21.15.37:0/1406121097 >> v1:172.21.15.37:6789/0 conn(0x7f0620048040 legacy=0x7f062004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7f06277fe640 1 -- 172.21.15.37:0/1406121097 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620053700 con 0x7f0620005e90 2024-04-26T20:21:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7fe9d4e54640 1 -- 172.21.15.37:0/2287380832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9c0004310 msgr2=0x7fe9c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7fe9d4e54640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9c0004310 0x7fe9c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7fe9d4e54640 1 -- 172.21.15.37:0/2287380832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c0004fe0 msgr2=0x7fe9c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7ffb8cb7e640 1 -- 172.21.15.37:0/1459199074 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1972329491 0 0) 0x7ffb74003520 con 0x7ffb880afae0 2024-04-26T20:21:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7ffb8db80640 1 -- 172.21.15.37:0/1459199074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb880cd7b0 msgr2=0x7ffb880cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7ffb8db80640 1 --2- 172.21.15.37:0/1459199074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb880cd7b0 0x7ffb880cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7ffb8db80640 1 -- 172.21.15.37:0/1459199074 >> v1:172.21.15.134:6789/0 conn(0x7ffb880afae0 legacy=0x7ffb881630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7ffb8db80640 1 -- 172.21.15.37:0/1459199074 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb881601a0 con 0x7ffb88159a70 2024-04-26T20:21:57.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.509+0000 7f6b546a0640 1 Processor -- start 2024-04-26T20:21:57.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7f6b546a0640 1 -- start start 2024-04-26T20:21:57.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7feb8f59e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feb90162d10 0x7feb90073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7feb8f59e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7feb90070b50 0x7feb900cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7feb8f59e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb901631a0 con 0x7feb9015f240 2024-04-26T20:21:57.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7feb8f59e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7feb9015f620 con 0x7feb90162d10 2024-04-26T20:21:57.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.510+0000 7feb8f59e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7feb90163480 con 0x7feb90070b50 2024-04-26T20:21:57.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7feb8e59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7feb9015f240 0x7feb90070440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44358/0 (socket says 172.21.15.37:44358) 2024-04-26T20:21:57.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7feb8e59c640 1 -- 172.21.15.37:0/4150041269 learned_addr learned my addr 172.21.15.37:0/4150041269 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:57.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413d395640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41300df220 0x7f4130038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413d395640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4130053c70 0x7f413004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413d395640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4130033ef0 0x7f413004ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413d395640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f41300544c0 con 0x7f4130053c70 2024-04-26T20:21:57.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7feb8ed9d640 1 --2- 172.21.15.37:0/4150041269 >> v2:172.21.15.134:3300/0 conn(0x7feb90070b50 0x7feb900cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413d395640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f41300541e0 con 0x7f4130033ef0 2024-04-26T20:21:57.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413d395640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4130005f30 con 0x7f41300df220 2024-04-26T20:21:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354412990 0 0) 0x7feb901631a0 con 0x7feb9015f240 2024-04-26T20:21:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb78003520 con 0x7feb9015f240 2024-04-26T20:21:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7feb8dd9b640 1 --2- 172.21.15.37:0/4150041269 >> v2:172.21.15.37:3300/0 conn(0x7feb90162d10 0x7feb90073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413cb94640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4130033ef0 0x7f413004ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413cb94640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4130033ef0 0x7f413004ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:33706/0 (socket says 172.21.15.37:33706) 2024-04-26T20:21:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413cb94640 1 -- 172.21.15.37:0/3836727622 learned_addr learned my addr 172.21.15.37:0/3836727622 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f4137fff640 1 --2- 172.21.15.37:0/3836727622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41300df220 0x7f4130038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413cb94640 1 -- 172.21.15.37:0/3836727622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41300df220 msgr2=0x7f4130038080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413cb94640 1 --2- 172.21.15.37:0/3836727622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41300df220 0x7f4130038080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413cb94640 1 -- 172.21.15.37:0/3836727622 >> v2:172.21.15.37:3300/0 conn(0x7f4130053c70 msgr2=0x7f413004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f41377fe640 1 --2- 172.21.15.37:0/3836727622 >> v2:172.21.15.37:3300/0 conn(0x7f4130053c70 0x7f413004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2517557295 0 0) 0x7feb78003520 con 0x7feb9015f240 2024-04-26T20:21:57.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 >> v2:172.21.15.134:3300/0 conn(0x7feb90070b50 msgr2=0x7feb900cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7feb8d59a640 1 --2- 172.21.15.37:0/4150041269 >> v2:172.21.15.134:3300/0 conn(0x7feb90070b50 0x7feb900cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 >> v2:172.21.15.37:3300/0 conn(0x7feb90162d10 msgr2=0x7feb90073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413cb94640 1 --2- 172.21.15.37:0/3836727622 >> v2:172.21.15.37:3300/0 conn(0x7f4130053c70 0x7f413004fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7f413cb94640 1 -- 172.21.15.37:0/3836727622 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4130006210 con 0x7f4130033ef0 2024-04-26T20:21:57.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7f41377fe640 1 --2- 172.21.15.37:0/3836727622 >> v2:172.21.15.37:3300/0 conn(0x7f4130053c70 0x7f413004fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:57.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8d59a640 1 --2- 172.21.15.37:0/4150041269 >> v2:172.21.15.37:3300/0 conn(0x7feb90162d10 0x7feb90073c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb90163760 con 0x7feb9015f240 2024-04-26T20:21:57.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7ffb8db80640 1 --2- 172.21.15.37:0/1459199074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb88159a70 0x7ffb880b3b90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009d90 tx=0x7ffb8002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6847f9d6f9e3a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7f6b546a0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6b4c15e860 0x7f6b4c15ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7fe9d4e54640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c0004fe0 0x7fe9c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.511+0000 7fe9d4e54640 1 -- 172.21.15.37:0/2287380832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c0003520 con 0x7fe9c00039b0 2024-04-26T20:21:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7feb84001670 con 0x7feb9015f240 2024-04-26T20:21:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 >> v1:172.21.15.37:6789/0 conn(0x7feb9015f240 legacy=0x7feb90070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8d59a640 1 --2- 172.21.15.37:0/4150041269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb780039b0 0x7feb78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8d59a640 1 --2- 172.21.15.37:0/4150041269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb78004800 0x7feb78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8d59a640 1 --2- 172.21.15.37:0/4150041269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb780051c0 0x7feb7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb90163480 con 0x7feb780051c0 2024-04-26T20:21:57.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb7800c040 con 0x7feb78004800 2024-04-26T20:21:57.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feb7800c240 con 0x7feb780039b0 2024-04-26T20:21:57.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8dd9b640 1 --2- 172.21.15.37:0/4150041269 >> v2:172.21.15.37:3300/0 conn(0x7feb90162d10 0x7feb90073c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7f6b546a0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6b4c06e0d0 0x7f6b4c06e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7f6b546a0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b4c073740 con 0x7f6b4c073280 2024-04-26T20:21:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7f6b546a0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6b4c073a20 con 0x7f6b4c06e0d0 2024-04-26T20:21:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7f6b546a0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6b4c073d00 con 0x7f6b4c15e860 2024-04-26T20:21:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7ffb8cb7e640 1 -- 172.21.15.37:0/1459199074 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84001670 con 0x7ffb88159a70 2024-04-26T20:21:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7ffb8cb7e640 1 -- 172.21.15.37:0/1459199074 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb80007540 con 0x7ffb88159a70 2024-04-26T20:21:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7ffb8cb7e640 1 -- 172.21.15.37:0/1459199074 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80007910 con 0x7ffb88159a70 2024-04-26T20:21:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7fe9d5655640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c0004fe0 0x7fe9c000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7fe9d5655640 1 -- 172.21.15.37:0/2287380832 reap_dead start 2024-04-26T20:21:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8e59c640 1 --2- 172.21.15.37:0/4150041269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb780039b0 0x7feb78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7feb8dd9b640 1 --2- 172.21.15.37:0/4150041269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb78004800 0x7feb78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7ff7913c9640 1 Processor -- start 2024-04-26T20:21:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7f6b52c3d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6b4c06e0d0 0x7f6b4c06e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7f6b5243c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6b4c15e860 0x7f6b4c15ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7f6b52c3d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6b4c06e0d0 0x7f6b4c06e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35110/0 (socket says 172.21.15.37:35110) 2024-04-26T20:21:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7f6b52c3d640 1 -- 172.21.15.37:0/465665616 learned_addr learned my addr 172.21.15.37:0/465665616 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7f06277fe640 1 --2- 172.21.15.37:0/1406121097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 0x7f06200062f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f061c0097a0 tx=0x7f061c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d783f0d8d06c3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7fe9d4e54640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 0x7fe9c0003d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc001890 tx=0x7fe9c0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ebbe034260b24dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.512+0000 7ff7913c9640 1 -- start start 2024-04-26T20:21:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7feb8e59c640 1 -- 172.21.15.37:0/4150041269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb78004800 msgr2=0x7feb78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc001460 con 0x7fe9c00039b0 2024-04-26T20:21:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9bc005310 con 0x7fe9c00039b0 2024-04-26T20:21:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7f6b52c3d640 1 -- 172.21.15.37:0/465665616 >> v2:172.21.15.134:3300/0 conn(0x7f6b4c15e860 msgr2=0x7f6b4c15ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7fe9cf7fe640 1 -- 172.21.15.37:0/2287380832 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc02ac40 con 0x7fe9c00039b0 2024-04-26T20:21:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7f6b52c3d640 1 --2- 172.21.15.37:0/465665616 >> v2:172.21.15.134:3300/0 conn(0x7f6b4c15e860 0x7f6b4c15ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7f6b52c3d640 1 -- 172.21.15.37:0/465665616 >> v1:172.21.15.37:6789/0 conn(0x7f6b4c073280 legacy=0x7f6b4c15f340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:21:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7f6b52c3d640 1 -- 172.21.15.37:0/465665616 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c073fe0 con 0x7f6b4c06e0d0 2024-04-26T20:21:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7f413cb94640 1 --2- 172.21.15.37:0/3836727622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4130033ef0 0x7f413004ffd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f413806adf0 tx=0x7f41380a6ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed4ca5f6d152539 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1459199074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb88159a70 msgr2=0x7ffb880b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7ffb8eb82640 1 --2- 172.21.15.37:0/1459199074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb88159a70 0x7ffb880b3b90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009d90 tx=0x7ffb8002a5f0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1459199074 shutdown_connections 2024-04-26T20:21:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7ffb8eb82640 1 --2- 172.21.15.37:0/1459199074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb880cd7b0 0x7ffb880cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7ffb8eb82640 1 --2- 172.21.15.37:0/1459199074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb88159a70 0x7ffb880b3b90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1459199074 >> 172.21.15.37:0/1459199074 conn(0x7ffb880d0ca0 msgr2=0x7ffb880d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7f4136ffd640 1 -- 172.21.15.37:0/3836727622 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41380a8900 con 0x7f4130033ef0 2024-04-26T20:21:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7f4136ffd640 1 -- 172.21.15.37:0/3836727622 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41380a8ae0 con 0x7f4130033ef0 2024-04-26T20:21:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1459199074 shutdown_connections 2024-04-26T20:21:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7feb8e59c640 1 --2- 172.21.15.37:0/4150041269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb78004800 0x7feb78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7feb8e59c640 1 -- 172.21.15.37:0/4150041269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb780051c0 msgr2=0x7feb7800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7feb8e59c640 1 --2- 172.21.15.37:0/4150041269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb780051c0 0x7feb7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7feb8e59c640 1 -- 172.21.15.37:0/4150041269 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb90163760 con 0x7feb780039b0 2024-04-26T20:21:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7feb8dd9b640 1 --2- 172.21.15.37:0/4150041269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb78004800 0x7feb78004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7feb8ed9d640 1 -- 172.21.15.37:0/4150041269 reap_dead start 2024-04-26T20:21:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7fe9d5e56640 1 -- 172.21.15.37:0/2287380832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 msgr2=0x7fe9c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7ff7913c9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff78c158ed0 0x7ff78c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7ff7913c9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff78c1592b0 con 0x7ff78c073f80 2024-04-26T20:21:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7ff7913c9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff78c0b6af0 con 0x7ff78c0b6610 2024-04-26T20:21:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7ff7913c9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff78c0b6dd0 con 0x7ff78c158ed0 2024-04-26T20:21:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7ff78b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff78c073f80 0x7ff78c15eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44372/0 (socket says 172.21.15.37:44372) 2024-04-26T20:21:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7ff78b7fe640 1 -- 172.21.15.37:0/2010925293 learned_addr learned my addr 172.21.15.37:0/2010925293 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7ff78affd640 1 --2- 172.21.15.37:0/2010925293 >> v2:172.21.15.134:3300/0 conn(0x7ff78c158ed0 0x7ff78c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7f0626ffd640 1 -- 172.21.15.37:0/1406121097 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c037070 con 0x7f0620005e90 2024-04-26T20:21:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7f0626ffd640 1 -- 172.21.15.37:0/1406121097 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c032440 con 0x7f0620005e90 2024-04-26T20:21:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.513+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1459199074 wait complete. 2024-04-26T20:21:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7f413d395640 1 -- 172.21.15.37:0/3836727622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4130033ef0 msgr2=0x7f413004ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7f413d395640 1 --2- 172.21.15.37:0/3836727622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4130033ef0 0x7f413004ffd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f413806adf0 tx=0x7f41380a6ae0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7fe9d5e56640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 0x7fe9c0003d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc001890 tx=0x7fe9c0005830 comp rx=0 tx=0).stop 2024-04-26T20:21:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7f6b52c3d640 1 --2- 172.21.15.37:0/465665616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b4c06e0d0 0x7f6b4c06e560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0b5da0 tx=0x7f6b4402b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab22dbbe4ead691 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7feb8e59c640 1 --2- 172.21.15.37:0/4150041269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb780039b0 0x7feb78003dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb8400c290 tx=0x7feb78005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc33e27273e3e16e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7f413d395640 1 -- 172.21.15.37:0/3836727622 shutdown_connections 2024-04-26T20:21:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7f413d395640 1 --2- 172.21.15.37:0/3836727622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4130033ef0 0x7f413004ffd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7f413d395640 1 --2- 172.21.15.37:0/3836727622 >> v2:172.21.15.37:3300/0 conn(0x7f4130053c70 0x7f413004fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7f413d395640 1 --2- 172.21.15.37:0/3836727622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41300df220 0x7f4130038080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7f413d395640 1 -- 172.21.15.37:0/3836727622 >> 172.21.15.37:0/3836727622 conn(0x7f4130009fb0 msgr2=0x7f413000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7ff789ffb640 1 -- 172.21.15.37:0/2010925293 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196420299 0 0) 0x7ff780004ad0 con 0x7ff78c073f80 2024-04-26T20:21:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7ff789ffb640 1 -- 172.21.15.37:0/2010925293 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff774003520 con 0x7ff78c073f80 2024-04-26T20:21:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7ff78affd640 1 -- 172.21.15.37:0/2010925293 >> v1:172.21.15.134:6789/0 conn(0x7ff78c0b6610 legacy=0x7ff78c073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7ff78affd640 1 -- 172.21.15.37:0/2010925293 >> v1:172.21.15.37:6789/0 conn(0x7ff78c073f80 legacy=0x7ff78c15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7ff78affd640 1 -- 172.21.15.37:0/2010925293 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c072d90 con 0x7ff78c158ed0 2024-04-26T20:21:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7f6b5143a640 1 -- 172.21.15.37:0/465665616 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44037070 con 0x7f6b4c06e0d0 2024-04-26T20:21:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f6b5143a640 1 -- 172.21.15.37:0/465665616 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b44032600 con 0x7f6b4c06e0d0 2024-04-26T20:21:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f6b5143a640 1 -- 172.21.15.37:0/465665616 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b440329f0 con 0x7f6b4c06e0d0 2024-04-26T20:21:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f0626ffd640 1 -- 172.21.15.37:0/1406121097 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c032810 con 0x7f0620005e90 2024-04-26T20:21:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f062d031640 1 -- 172.21.15.37:0/1406121097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 msgr2=0x7f06200062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f062d031640 1 --2- 172.21.15.37:0/1406121097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 0x7f06200062f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f061c0097a0 tx=0x7f061c02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f062d031640 1 -- 172.21.15.37:0/1406121097 shutdown_connections 2024-04-26T20:21:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f062d031640 1 --2- 172.21.15.37:0/1406121097 >> v2:172.21.15.37:3300/0 conn(0x7f0620006830 0x7f0620006cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f062d031640 1 --2- 172.21.15.37:0/1406121097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 0x7f06200062f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f062d031640 1 -- 172.21.15.37:0/1406121097 >> 172.21.15.37:0/1406121097 conn(0x7f0620009fb0 msgr2=0x7f062000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f062d031640 1 -- 172.21.15.37:0/1406121097 shutdown_connections 2024-04-26T20:21:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7ffb8eb82640 1 Processor -- start 2024-04-26T20:21:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f413d395640 1 -- 172.21.15.37:0/3836727622 shutdown_connections 2024-04-26T20:21:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7fe9d5e56640 1 -- 172.21.15.37:0/2287380832 shutdown_connections 2024-04-26T20:21:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7fe9d5e56640 1 --2- 172.21.15.37:0/2287380832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 0x7fe9c0003d90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7fe9d5e56640 1 -- 172.21.15.37:0/2287380832 >> 172.21.15.37:0/2287380832 conn(0x7fe9c8055350 msgr2=0x7fe9c8055760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7fe9d5e56640 1 -- 172.21.15.37:0/2287380832 shutdown_connections 2024-04-26T20:21:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84036070 con 0x7feb780039b0 2024-04-26T20:21:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb84005ad0 con 0x7feb780039b0 2024-04-26T20:21:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.514+0000 7feb8d59a640 1 -- 172.21.15.37:0/4150041269 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84007d80 con 0x7feb780039b0 2024-04-26T20:21:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7ffb8eb82640 1 -- start start 2024-04-26T20:21:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7feb8f59e640 1 -- 172.21.15.37:0/4150041269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb780039b0 msgr2=0x7feb78003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7feb8f59e640 1 --2- 172.21.15.37:0/4150041269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb780039b0 0x7feb78003dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb8400c290 tx=0x7feb78005830 comp rx=0 tx=0).stop 2024-04-26T20:21:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7feb8f59e640 1 -- 172.21.15.37:0/4150041269 shutdown_connections 2024-04-26T20:21:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7f6b546a0640 1 -- 172.21.15.37:0/465665616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b4c06e0d0 msgr2=0x7f6b4c06e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7f6b546a0640 1 --2- 172.21.15.37:0/465665616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b4c06e0d0 0x7f6b4c06e560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0b5da0 tx=0x7f6b4402b800 comp rx=0 tx=0).stop 2024-04-26T20:21:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7fe9d5e56640 1 -- 172.21.15.37:0/2287380832 wait complete. 2024-04-26T20:21:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.515+0000 7f062d031640 1 -- 172.21.15.37:0/1406121097 wait complete. 2024-04-26T20:21:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7feb8f59e640 1 --2- 172.21.15.37:0/4150041269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb780039b0 0x7feb78003dd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7f6b546a0640 1 -- 172.21.15.37:0/465665616 shutdown_connections 2024-04-26T20:21:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7f413d395640 1 -- 172.21.15.37:0/3836727622 wait complete. 2024-04-26T20:21:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7f6b546a0640 1 --2- 172.21.15.37:0/465665616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b4c06e0d0 0x7f6b4c06e560 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7f6b546a0640 1 --2- 172.21.15.37:0/465665616 >> v2:172.21.15.134:3300/0 conn(0x7f6b4c15e860 0x7f6b4c15ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7f6b546a0640 1 -- 172.21.15.37:0/465665616 >> 172.21.15.37:0/465665616 conn(0x7f6b4c0b2890 msgr2=0x7f6b4c0b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7ff78affd640 1 --2- 172.21.15.37:0/2010925293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff78c158ed0 0x7ff78c0ce920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff78400be90 tx=0x7ff78402c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=568386c44ffce1a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7feb8f59e640 1 -- 172.21.15.37:0/4150041269 >> 172.21.15.37:0/4150041269 conn(0x7feb9006d350 msgr2=0x7feb9006d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7f6b546a0640 1 -- 172.21.15.37:0/465665616 shutdown_connections 2024-04-26T20:21:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7feb8f59e640 1 -- 172.21.15.37:0/4150041269 shutdown_connections 2024-04-26T20:21:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7ff789ffb640 1 -- 172.21.15.37:0/2010925293 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78403a070 con 0x7ff78c158ed0 2024-04-26T20:21:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7ff789ffb640 1 -- 172.21.15.37:0/2010925293 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff784032480 con 0x7ff78c158ed0 2024-04-26T20:21:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7ff789ffb640 1 -- 172.21.15.37:0/2010925293 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784032850 con 0x7ff78c158ed0 2024-04-26T20:21:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7f6b546a0640 1 -- 172.21.15.37:0/465665616 wait complete. 2024-04-26T20:21:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7fe9d5e56640 1 Processor -- start 2024-04-26T20:21:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7fe9d5e56640 1 -- start start 2024-04-26T20:21:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8eb82640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb880afae0 0x7ffb8814f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8eb82640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb880cd7b0 0x7ffb88152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8eb82640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb88159a70 0x7ffb88072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8eb82640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8815f980 con 0x7ffb880cd7b0 2024-04-26T20:21:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8eb82640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8815f800 con 0x7ffb880afae0 2024-04-26T20:21:57.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8eb82640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffb880aff70 con 0x7ffb88159a70 2024-04-26T20:21:57.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8d37f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb880cd7b0 0x7ffb88152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8d37f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb880cd7b0 0x7ffb88152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35132/0 (socket says 172.21.15.37:35132) 2024-04-26T20:21:57.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8d37f640 1 -- 172.21.15.37:0/1505082847 learned_addr learned my addr 172.21.15.37:0/1505082847 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:57.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8e381640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb88159a70 0x7ffb88072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7f062d031640 1 Processor -- start 2024-04-26T20:21:57.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8db80640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb880afae0 0x7ffb8814f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8d37f640 1 -- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb88159a70 msgr2=0x7ffb88072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8d37f640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb88159a70 0x7ffb88072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8d37f640 1 -- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb880afae0 msgr2=0x7ffb8814f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8d37f640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb880afae0 0x7ffb8814f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.517+0000 7ffb8d37f640 1 -- 172.21.15.37:0/1505082847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb88153590 con 0x7ffb880cd7b0 2024-04-26T20:21:57.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7ffb8e381640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb88159a70 0x7ffb88072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:57.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7f413d395640 1 Processor -- start 2024-04-26T20:21:57.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7ffb8d37f640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb880cd7b0 0x7ffb88152c20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800d800 tx=0x7ffb7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7f413d395640 1 -- start start 2024-04-26T20:21:57.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7fe9d5e56640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 0x7fe9c80d5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7fe9d5e56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c80ea180 0x7fe9c80d38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7fe9d5e56640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9c81804f0 0x7fe9c8180970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7fe9d4e54640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 0x7fe9c80d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.516+0000 7feb8f59e640 1 -- 172.21.15.37:0/4150041269 wait complete. 2024-04-26T20:21:57.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7feb8f59e640 1 Processor -- start 2024-04-26T20:21:57.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7fe9d4e54640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 0x7fe9c80d5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:33746/0 (socket says 172.21.15.37:33746) 2024-04-26T20:21:57.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7fe9cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c80ea180 0x7fe9c80d38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7feb8f59e640 1 -- start start 2024-04-26T20:21:57.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7fe9cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c80ea180 0x7fe9c80d38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35140/0 (socket says 172.21.15.37:35140) 2024-04-26T20:21:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7fe9d5e56640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8054e50 con 0x7fe9c80ea180 2024-04-26T20:21:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7fe9d5e56640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8054cd0 con 0x7fe9c00039b0 2024-04-26T20:21:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7fe9d5e56640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c8054fd0 con 0x7fe9c81804f0 2024-04-26T20:21:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7f6b546a0640 1 Processor -- start 2024-04-26T20:21:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7fe9d4e54640 1 -- 172.21.15.37:0/2602800310 learned_addr learned my addr 172.21.15.37:0/2602800310 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7f6b546a0640 1 -- start start 2024-04-26T20:21:57.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7fe9d5655640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9c81804f0 0x7fe9c8180970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7ffb7e7fc640 1 -- 172.21.15.37:0/1505082847 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800b6c0 con 0x7ffb880cd7b0 2024-04-26T20:21:57.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7ffb7e7fc640 1 -- 172.21.15.37:0/1505082847 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb78019070 con 0x7ffb880cd7b0 2024-04-26T20:21:57.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7ffb7e7fc640 1 -- 172.21.15.37:0/1505082847 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800ba80 con 0x7ffb880cd7b0 2024-04-26T20:21:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7fe9cffff640 1 -- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9c81804f0 msgr2=0x7fe9c8180970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7fe9cffff640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9c81804f0 0x7fe9c8180970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7fe9cffff640 1 -- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 msgr2=0x7fe9c80d5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7fe9cffff640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 0x7fe9c80d5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7fe9cffff640 1 -- 172.21.15.37:0/2602800310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c80ea600 con 0x7fe9c80ea180 2024-04-26T20:21:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1505082847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb88153870 con 0x7ffb880cd7b0 2024-04-26T20:21:57.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7f413d395640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4130033ef0 0x7f41300d0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7f413d395640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4130053c70 0x7f41300e1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7f413d395640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41300df220 0x7f41300e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.519+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1505082847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb88153e30 con 0x7ffb880cd7b0 2024-04-26T20:21:57.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f413d395640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4130037a90 con 0x7f4130053c70 2024-04-26T20:21:57.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f4137fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4130033ef0 0x7f41300d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f413d395640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4130037910 con 0x7f4130033ef0 2024-04-26T20:21:57.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f413d395640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f41300df600 con 0x7f41300df220 2024-04-26T20:21:57.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f4137fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4130033ef0 0x7f41300d0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:33762/0 (socket says 172.21.15.37:33762) 2024-04-26T20:21:57.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f4137fff640 1 -- 172.21.15.37:0/1599515611 learned_addr learned my addr 172.21.15.37:0/1599515611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:57.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f413cb94640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41300df220 0x7f41300e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7ff7913c9640 1 -- 172.21.15.37:0/2010925293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff78c158ed0 msgr2=0x7ff78c0ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7ff7913c9640 1 --2- 172.21.15.37:0/2010925293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff78c158ed0 0x7ff78c0ce920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff78400be90 tx=0x7ff78402c7a0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f41377fe640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4130053c70 0x7f41300e1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.518+0000 7f062d031640 1 -- start start 2024-04-26T20:21:57.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f062d031640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 0x7f06200d6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f062d031640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0620006830 0x7f06200e9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f062d031640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0620048040 0x7f06200ea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f062d031640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0620053fa0 con 0x7f0620006830 2024-04-26T20:21:57.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f062d031640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0620053cc0 con 0x7f0620048040 2024-04-26T20:21:57.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f062d031640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0620054280 con 0x7f0620005e90 2024-04-26T20:21:57.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7fe9cffff640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c80ea180 0x7fe9c80d38f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0000b60 tx=0x7fe9d00747b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7ff7913c9640 1 -- 172.21.15.37:0/2010925293 shutdown_connections 2024-04-26T20:21:57.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7ff7913c9640 1 --2- 172.21.15.37:0/2010925293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff78c158ed0 0x7ff78c0ce920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7ff7913c9640 1 -- 172.21.15.37:0/2010925293 >> 172.21.15.37:0/2010925293 conn(0x7ff78c0d0d50 msgr2=0x7ff78c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f4137fff640 1 -- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41300df220 msgr2=0x7f41300e21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f4137fff640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41300df220 0x7f41300e21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f4137fff640 1 -- 172.21.15.37:0/1599515611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4130053c70 msgr2=0x7f41300e1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f4137fff640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4130053c70 0x7f41300e1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f4137fff640 1 -- 172.21.15.37:0/1599515611 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41300e06e0 con 0x7f4130033ef0 2024-04-26T20:21:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7feb8f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb780039b0 0x7feb9015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7feb8f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb9015fca0 0x7feb90160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7fe9cd7fa640 1 -- 172.21.15.37:0/2602800310 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0092070 con 0x7fe9c80ea180 2024-04-26T20:21:57.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7feb8f59e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb9015d460 0x7feb9015d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f06277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0620006830 0x7f06200e9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f0627fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 0x7f06200d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f062c830640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0620048040 0x7f06200ea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f0627fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 0x7f06200d6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:55764/0 (socket says 172.21.15.37:55764) 2024-04-26T20:21:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7f0627fff640 1 -- 172.21.15.37:0/2217723383 learned_addr learned my addr 172.21.15.37:0/2217723383 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7fe9d5e56640 1 -- 172.21.15.37:0/2602800310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c80d5ce0 con 0x7fe9c80ea180 2024-04-26T20:21:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8f59e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb900b14c0 con 0x7feb780039b0 2024-04-26T20:21:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7f4137fff640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4130033ef0 0x7f41300d0080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f412c016e70 tx=0x7f412c008480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8f59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb900b1340 con 0x7feb9015fca0 2024-04-26T20:21:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8f59e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feb90073c40 con 0x7feb9015d460 2024-04-26T20:21:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7f6b546a0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b4c06e0d0 0x7f6b4c1ee490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7f6b546a0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b4c073280 0x7f6b4c151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7f6b546a0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b4c15e860 0x7f6b4c150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7f6b546a0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c15fa50 con 0x7f6b4c06e0d0 2024-04-26T20:21:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7f6b546a0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c0d04b0 con 0x7f6b4c15e860 2024-04-26T20:21:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7f6b546a0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6b4c15fbd0 con 0x7f6b4c073280 2024-04-26T20:21:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7ff7913c9640 1 -- 172.21.15.37:0/2010925293 shutdown_connections 2024-04-26T20:21:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.520+0000 7ff7913c9640 1 -- 172.21.15.37:0/2010925293 wait complete. 2024-04-26T20:21:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.522+0000 7f4134ff9640 1 -- 172.21.15.37:0/1599515611 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c01f440 con 0x7f4130033ef0 2024-04-26T20:21:57.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.522+0000 7f4134ff9640 1 -- 172.21.15.37:0/1599515611 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412c01f5e0 con 0x7f4130033ef0 2024-04-26T20:21:57.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.522+0000 7f4134ff9640 1 -- 172.21.15.37:0/1599515611 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c01f9d0 con 0x7f4130033ef0 2024-04-26T20:21:57.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7fe9cd7fa640 1 -- 172.21.15.37:0/2602800310 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d0072b10 con 0x7fe9c80ea180 2024-04-26T20:21:57.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7fe9d5e56640 1 -- 172.21.15.37:0/2602800310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c80d62a0 con 0x7fe9c80ea180 2024-04-26T20:21:57.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.522+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1505082847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb88069030 con 0x7ffb880cd7b0 2024-04-26T20:21:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7fe9cd7fa640 1 -- 172.21.15.37:0/2602800310 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0097460 con 0x7fe9c80ea180 2024-04-26T20:21:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8ed9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb9015d460 0x7feb9015d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8dd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb9015fca0 0x7feb90160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8dd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb9015fca0 0x7feb90160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:33780/0 (socket says 172.21.15.37:33780) 2024-04-26T20:21:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8dd9b640 1 -- 172.21.15.37:0/2387206901 learned_addr learned my addr 172.21.15.37:0/2387206901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8dd9b640 1 -- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb9015d460 msgr2=0x7feb9015d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8dd9b640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb9015d460 0x7feb9015d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8dd9b640 1 -- 172.21.15.37:0/2387206901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb780039b0 msgr2=0x7feb9015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8dd9b640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb780039b0 0x7feb9015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.521+0000 7feb8dd9b640 1 -- 172.21.15.37:0/2387206901 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb90161e40 con 0x7feb9015fca0 2024-04-26T20:21:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.522+0000 7feb8dd9b640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb9015fca0 0x7feb90160120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb800118d0 tx=0x7feb80011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.522+0000 7f413d395640 1 -- 172.21.15.37:0/1599515611 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41300e09c0 con 0x7f4130033ef0 2024-04-26T20:21:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.522+0000 7f413d395640 1 -- 172.21.15.37:0/1599515611 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41300e0f80 con 0x7f4130033ef0 2024-04-26T20:21:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7ffb7e7fc640 1 -- 172.21.15.37:0/1505082847 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb7800bc20 con 0x7ffb880cd7b0 2024-04-26T20:21:57.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7f413d395640 1 -- 172.21.15.37:0/1599515611 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41300d3390 con 0x7f4130033ef0 2024-04-26T20:21:57.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7f06277fe640 1 -- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 msgr2=0x7f06200d6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7f06277fe640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 0x7f06200d6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7f06277fe640 1 -- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0620048040 msgr2=0x7f06200ea3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7f06277fe640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0620048040 0x7f06200ea3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7f06277fe640 1 -- 172.21.15.37:0/2217723383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062004d530 con 0x7f0620006830 2024-04-26T20:21:57.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7f6b52c3d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b4c15e860 0x7f6b4c150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7feb7effd640 1 -- 172.21.15.37:0/2387206901 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80019070 con 0x7feb9015fca0 2024-04-26T20:21:57.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7f6b52c3d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b4c15e860 0x7f6b4c150500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:33784/0 (socket says 172.21.15.37:33784) 2024-04-26T20:21:57.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7f6b52c3d640 1 -- 172.21.15.37:0/551420981 learned_addr learned my addr 172.21.15.37:0/551420981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:57.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7f6b51c3b640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b4c073280 0x7f6b4c151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.522+0000 7ff7913c9640 1 Processor -- start 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7ff7913c9640 1 -- start start 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7ff7913c9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff78c073f80 0x7ff78c158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7ff7913c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff78c0b6610 0x7ff78c1f9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7ff7913c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff78c158ed0 0x7ff78c0af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7ff7913c9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c0743f0 con 0x7ff78c073f80 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7ff7913c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c073350 con 0x7ff78c0b6610 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7ff7913c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff78c06e410 con 0x7ff78c158ed0 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7ff78b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff78c158ed0 0x7ff78c0af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7ff78a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff78c0b6610 0x7ff78c1f9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7ff78b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff78c158ed0 0x7ff78c0af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:55778/0 (socket says 172.21.15.37:55778) 2024-04-26T20:21:57.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7ff78b7fe640 1 -- 172.21.15.37:0/2993108301 learned_addr learned my addr 172.21.15.37:0/2993108301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:57.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7f6b5243c640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b4c06e0d0 0x7f6b4c1ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7f6b52c3d640 1 -- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b4c073280 msgr2=0x7f6b4c151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7f6b52c3d640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b4c073280 0x7f6b4c151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7f6b52c3d640 1 -- 172.21.15.37:0/551420981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b4c06e0d0 msgr2=0x7f6b4c1ee490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7f6b52c3d640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b4c06e0d0 0x7f6b4c1ee490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7f6b52c3d640 1 -- 172.21.15.37:0/551420981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c150c70 con 0x7f6b4c15e860 2024-04-26T20:21:57.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7f06277fe640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0620006830 0x7f06200e9e90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f061c031000 tx=0x7f061c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7f062c830640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0620048040 0x7f06200ea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:57.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7f0627fff640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 0x7f06200d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:57.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7ffb7e7fc640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb6c06c920 0x7ffb6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7ffb7e7fc640 1 -- 172.21.15.37:0/1505082847 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffb7808bec0 con 0x7ffb880cd7b0 2024-04-26T20:21:57.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7feb8f59e640 1 -- 172.21.15.37:0/2387206901 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb90162120 con 0x7feb9015fca0 2024-04-26T20:21:57.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.523+0000 7feb8f59e640 1 -- 172.21.15.37:0/2387206901 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb901626e0 con 0x7feb9015fca0 2024-04-26T20:21:57.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7feb7effd640 1 -- 172.21.15.37:0/2387206901 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb8000fbd0 con 0x7feb9015fca0 2024-04-26T20:21:57.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7f6b52c3d640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b4c15e860 0x7f6b4c150500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6b4402bdf0 tx=0x7f6b44004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7fe9baffd640 1 -- 172.21.15.37:0/2602800310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b0005290 con 0x7fe9c80ea180 2024-04-26T20:21:57.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.524+0000 7fe9cd7fa640 1 -- 172.21.15.37:0/2602800310 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9d006f050 con 0x7fe9c80ea180 2024-04-26T20:21:57.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7ffb8db80640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb6c06c920 0x7ffb6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7feb7effd640 1 -- 172.21.15.37:0/2387206901 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80009e20 con 0x7feb9015fca0 2024-04-26T20:21:57.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7ff78a7fc640 1 -- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff78c158ed0 msgr2=0x7ff78c0af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7ff78a7fc640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff78c158ed0 0x7ff78c0af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7ff78a7fc640 1 -- 172.21.15.37:0/2993108301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff78c073f80 msgr2=0x7ff78c158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7ff78a7fc640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff78c073f80 0x7ff78c158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7ff78a7fc640 1 -- 172.21.15.37:0/2993108301 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c0b03e0 con 0x7ff78c0b6610 2024-04-26T20:21:57.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7fe9cd7fa640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe9b406c9f0 0x7fe9b406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7fe9cd7fa640 1 -- 172.21.15.37:0/2602800310 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe9d0108c40 con 0x7fe9c80ea180 2024-04-26T20:21:57.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7feb8f59e640 1 -- 172.21.15.37:0/2387206901 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb74005290 con 0x7feb9015fca0 2024-04-26T20:21:57.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7ff78a7fc640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff78c0b6610 0x7ff78c1f9ec0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00c3e0 tx=0x7ff77c00c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7ffb8db80640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb6c06c920 0x7ffb6c06ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffb80034000 tx=0x7ffb8002f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.525+0000 7fe9d4e54640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe9b406c9f0 0x7fe9b406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.526+0000 7f4134ff9640 1 -- 172.21.15.37:0/1599515611 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f412c028440 con 0x7f4130033ef0 2024-04-26T20:21:57.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.526+0000 7fe9d4e54640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe9b406c9f0 0x7fe9b406eeb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc000fb0 tx=0x7fe9bc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.526+0000 7f4134ff9640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f411c06c940 0x7f411c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.526+0000 7f4134ff9640 1 -- 172.21.15.37:0/1599515611 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f412c094130 con 0x7f4130033ef0 2024-04-26T20:21:57.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.526+0000 7f41377fe640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f411c06c940 0x7f411c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.527+0000 7f41377fe640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f411c06c940 0x7f411c06ee00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4130034400 tx=0x7f4124008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.529+0000 7f4134ff9640 1 -- 172.21.15.37:0/1599515611 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f412c0626d0 con 0x7f4130033ef0 2024-04-26T20:21:57.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.529+0000 7ff7737fe640 1 -- 172.21.15.37:0/2993108301 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c011790 con 0x7ff78c0b6610 2024-04-26T20:21:57.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.529+0000 7f6b42ffd640 1 -- 172.21.15.37:0/551420981 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44037050 con 0x7f6b4c15e860 2024-04-26T20:21:57.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.529+0000 7f6b42ffd640 1 -- 172.21.15.37:0/551420981 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b44041070 con 0x7f6b4c15e860 2024-04-26T20:21:57.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.529+0000 7f6b546a0640 1 -- 172.21.15.37:0/551420981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4c150f50 con 0x7f6b4c15e860 2024-04-26T20:21:57.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.529+0000 7ff7737fe640 1 -- 172.21.15.37:0/2993108301 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77c017070 con 0x7ff78c0b6610 2024-04-26T20:21:57.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.529+0000 7ff7913c9640 1 -- 172.21.15.37:0/2993108301 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78c0afe20 con 0x7ff78c0b6610 2024-04-26T20:21:57.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.530+0000 7ff7737fe640 1 -- 172.21.15.37:0/2993108301 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c011b70 con 0x7ff78c0b6610 2024-04-26T20:21:57.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.530+0000 7feb7effd640 1 -- 172.21.15.37:0/2387206901 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb8001c070 con 0x7feb9015fca0 2024-04-26T20:21:57.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.530+0000 7ff7913c9640 1 -- 172.21.15.37:0/2993108301 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78c1f96c0 con 0x7ff78c0b6610 2024-04-26T20:21:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.530+0000 7f6b42ffd640 1 -- 172.21.15.37:0/551420981 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44008b40 con 0x7f6b4c15e860 2024-04-26T20:21:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.530+0000 7f6b546a0640 1 -- 172.21.15.37:0/551420981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b4c151510 con 0x7f6b4c15e860 2024-04-26T20:21:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.530+0000 7feb7effd640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feb7006c940 0x7feb7006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.530+0000 7feb7effd640 1 -- 172.21.15.37:0/2387206901 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feb8008c540 con 0x7feb9015fca0 2024-04-26T20:21:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.531+0000 7f0624ff9640 1 -- 172.21.15.37:0/2217723383 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c037050 con 0x7f0620006830 2024-04-26T20:21:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.531+0000 7f062d031640 1 -- 172.21.15.37:0/2217723383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06200d70d0 con 0x7f0620006830 2024-04-26T20:21:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.531+0000 7f062d031640 1 -- 172.21.15.37:0/2217723383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06200d7690 con 0x7f0620006830 2024-04-26T20:21:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.531+0000 7ffb7e7fc640 1 -- 172.21.15.37:0/1505082847 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb78059c90 con 0x7ffb880cd7b0 2024-04-26T20:21:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.531+0000 7feb8e59c640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feb7006c940 0x7feb7006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.531+0000 7ff7913c9640 1 -- 172.21.15.37:0/2993108301 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff78c06a360 con 0x7ff78c0b6610 2024-04-26T20:21:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.531+0000 7feb8e59c640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feb7006c940 0x7feb7006ee00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb84033000 tx=0x7feb84007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.531+0000 7f6b546a0640 1 -- 172.21.15.37:0/551420981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b4c0b5bc0 con 0x7f6b4c15e860 2024-04-26T20:21:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.532+0000 7f062d031640 1 -- 172.21.15.37:0/2217723383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06200089d0 con 0x7f0620006830 2024-04-26T20:21:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.533+0000 7feb7effd640 1 -- 172.21.15.37:0/2387206901 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb8005a300 con 0x7feb9015fca0 2024-04-26T20:21:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.535+0000 7ff7737fe640 1 -- 172.21.15.37:0/2993108301 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff77c011d10 con 0x7ff78c0b6610 2024-04-26T20:21:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.535+0000 7f6b42ffd640 1 -- 172.21.15.37:0/551420981 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b44008ce0 con 0x7f6b4c15e860 2024-04-26T20:21:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.536+0000 7f6b42ffd640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6b3006c920 0x7f6b3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.536+0000 7ff7737fe640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff77806c8f0 0x7ff77806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.536+0000 7f6b42ffd640 1 -- 172.21.15.37:0/551420981 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6b440acf90 con 0x7f6b4c15e860 2024-04-26T20:21:57.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.536+0000 7f6b5243c640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6b3006c920 0x7f6b3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.536+0000 7ff78affd640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff77806c8f0 0x7ff77806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.536+0000 7ff7737fe640 1 -- 172.21.15.37:0/2993108301 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff77c08c0b0 con 0x7ff78c0b6610 2024-04-26T20:21:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.537+0000 7f6b5243c640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6b3006c920 0x7f6b3006ede0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b48006fd0 tx=0x7f6b48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.537+0000 7ff78affd640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff77806c8f0 0x7ff77806edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff784037000 tx=0x7ff784032000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.538+0000 7ff7737fe640 1 -- 172.21.15.37:0/2993108301 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff77c021020 con 0x7ff78c0b6610 2024-04-26T20:21:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.538+0000 7f0624ff9640 1 -- 172.21.15.37:0/2217723383 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c038040 con 0x7f0620006830 2024-04-26T20:21:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.538+0000 7f0624ff9640 1 -- 172.21.15.37:0/2217723383 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c03c810 con 0x7f0620006830 2024-04-26T20:21:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.538+0000 7fe9cd7fa640 1 -- 172.21.15.37:0/2602800310 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe9d00d70e0 con 0x7fe9c80ea180 2024-04-26T20:21:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.541+0000 7f0624ff9640 1 -- 172.21.15.37:0/2217723383 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f061c045020 con 0x7f0620006830 2024-04-26T20:21:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.541+0000 7f0624ff9640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f060c06c8f0 0x7f060c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.542+0000 7f0624ff9640 1 -- 172.21.15.37:0/2217723383 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f061c0ac7a0 con 0x7f0620006830 2024-04-26T20:21:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.542+0000 7f0627fff640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f060c06c8f0 0x7f060c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.542+0000 7f0627fff640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f060c06c8f0 0x7f060c06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0628065980 tx=0x7f062806a160 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.543+0000 7f6b42ffd640 1 -- 172.21.15.37:0/551420981 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b4c0b5bc0 con 0x7f6b4c15e860 2024-04-26T20:21:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.550+0000 7f0624ff9640 1 -- 172.21.15.37:0/2217723383 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f061c04c020 con 0x7f0620006830 2024-04-26T20:21:57.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.903+0000 7efd9aac2640 1 -- 172.21.15.37:0/1837195955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efd7c005580 con 0x7efd8c0e00a0 2024-04-26T20:21:57.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.904+0000 7efd8a7fc640 1 -- 172.21.15.37:0/1837195955 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efd90079f10 con 0x7efd8c0e00a0 2024-04-26T20:21:57.905 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:57.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.907+0000 7efd77fff640 1 -- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efd7806c9c0 msgr2=0x7efd7806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.908+0000 7efd77fff640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efd7806c9c0 0x7efd7806ee80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd84006fd0 tx=0x7efd84008040 comp rx=0 tx=0).stop 2024-04-26T20:21:57.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.908+0000 7efd77fff640 1 -- 172.21.15.37:0/1837195955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c0e00a0 msgr2=0x7efd8c188ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.908+0000 7efd77fff640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c0e00a0 0x7efd8c188ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efd8c050ab0 tx=0x7efd900046c0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.908+0000 7efd77fff640 1 -- 172.21.15.37:0/1837195955 shutdown_connections 2024-04-26T20:21:57.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.908+0000 7efd77fff640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd8c0e1fd0 0x7efd8c0e2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.908+0000 7efd77fff640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c0e00a0 0x7efd8c188ce0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.908+0000 7efd77fff640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efd7806c9c0 0x7efd7806ee80 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd84006fd0 tx=0x7efd84008040 comp rx=0 tx=0).stop 2024-04-26T20:21:57.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.909+0000 7efd77fff640 1 --2- 172.21.15.37:0/1837195955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd8c039320 0x7efd8c17a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.909+0000 7efd77fff640 1 -- 172.21.15.37:0/1837195955 >> 172.21.15.37:0/1837195955 conn(0x7efd8c055bd0 msgr2=0x7efd8c04e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.909+0000 7efd77fff640 1 -- 172.21.15.37:0/1837195955 shutdown_connections 2024-04-26T20:21:57.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.909+0000 7efd77fff640 1 -- 172.21.15.37:0/1837195955 wait complete. 2024-04-26T20:21:57.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-04-26T20:21:57.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.951+0000 7fe9baffd640 1 -- 172.21.15.37:0/2602800310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe9b0005580 con 0x7fe9c80ea180 2024-04-26T20:21:57.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.951+0000 7fe9cd7fa640 1 -- 172.21.15.37:0/2602800310 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe9d00d69e0 con 0x7fe9c80ea180 2024-04-26T20:21:57.953 INFO:teuthology.orchestra.run.smithi037.stdout:55834574854 2024-04-26T20:21:57.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 -- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe9b406c9f0 msgr2=0x7fe9b406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe9b406c9f0 0x7fe9b406eeb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc000fb0 tx=0x7fe9bc02e000 comp rx=0 tx=0).stop 2024-04-26T20:21:57.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 -- 172.21.15.37:0/2602800310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c80ea180 msgr2=0x7fe9c80d38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c80ea180 0x7fe9c80d38f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0000b60 tx=0x7fe9d00747b0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 -- 172.21.15.37:0/2602800310 shutdown_connections 2024-04-26T20:21:57.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe9b406c9f0 0x7fe9b406eeb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9c81804f0 0x7fe9c8180970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c80ea180 0x7fe9c80d38f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 --2- 172.21.15.37:0/2602800310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c00039b0 0x7fe9c80d5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 -- 172.21.15.37:0/2602800310 >> 172.21.15.37:0/2602800310 conn(0x7fe9c8055350 msgr2=0x7fe9c803a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 -- 172.21.15.37:0/2602800310 shutdown_connections 2024-04-26T20:21:57.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.954+0000 7fe9baffd640 1 -- 172.21.15.37:0/2602800310 wait complete. 2024-04-26T20:21:57.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-04-26T20:21:57.967 DEBUG:teuthology.parallel:result is None 2024-04-26T20:21:57.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.972+0000 7f6b546a0640 1 -- 172.21.15.37:0/551420981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6b4c076900 con 0x7f6b4c15e860 2024-04-26T20:21:57.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.973+0000 7f6b42ffd640 1 -- 172.21.15.37:0/551420981 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6b4407b430 con 0x7f6b4c15e860 2024-04-26T20:21:57.974 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.973+0000 7f413d395640 1 -- 172.21.15.37:0/1599515611 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4130052be0 con 0x7f4130033ef0 2024-04-26T20:21:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.973+0000 7f4134ff9640 1 -- 172.21.15.37:0/1599515611 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f412c061fd0 con 0x7f4130033ef0 2024-04-26T20:21:57.974 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.974+0000 7feb8f59e640 1 -- 172.21.15.37:0/2387206901 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7feb74005850 con 0x7feb9015fca0 2024-04-26T20:21:57.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 -- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6b3006c920 msgr2=0x7f6b3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6b3006c920 0x7f6b3006ede0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b48006fd0 tx=0x7f6b48008040 comp rx=0 tx=0).stop 2024-04-26T20:21:57.976 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:57.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7feb7effd640 1 -- 172.21.15.37:0/2387206901 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feb8005e600 con 0x7feb9015fca0 2024-04-26T20:21:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 -- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b4c15e860 msgr2=0x7f6b4c150500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b4c15e860 0x7f6b4c150500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6b4402bdf0 tx=0x7f6b44004a40 comp rx=0 tx=0).stop 2024-04-26T20:21:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 -- 172.21.15.37:0/551420981 shutdown_connections 2024-04-26T20:21:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b4c15e860 0x7f6b4c150500 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6b3006c920 0x7f6b3006ede0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b4c073280 0x7f6b4c151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 --2- 172.21.15.37:0/551420981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b4c06e0d0 0x7f6b4c1ee490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 -- 172.21.15.37:0/551420981 >> 172.21.15.37:0/551420981 conn(0x7f6b4c0b2890 msgr2=0x7f6b4c0d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 -- 172.21.15.37:0/551420981 shutdown_connections 2024-04-26T20:21:57.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.975+0000 7f6b546a0640 1 -- 172.21.15.37:0/551420981 wait complete. 2024-04-26T20:21:57.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 -- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f411c06c940 msgr2=0x7f411c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f411c06c940 0x7f411c06ee00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4130034400 tx=0x7f4124008040 comp rx=0 tx=0).stop 2024-04-26T20:21:57.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 -- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4130033ef0 msgr2=0x7f41300d0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4130033ef0 0x7f41300d0080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f412c016e70 tx=0x7f412c008480 comp rx=0 tx=0).stop 2024-04-26T20:21:57.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 -- 172.21.15.37:0/1599515611 shutdown_connections 2024-04-26T20:21:57.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f411c06c940 0x7f411c06ee00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41300df220 0x7f41300e21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4130053c70 0x7f41300e1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 --2- 172.21.15.37:0/1599515611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4130033ef0 0x7f41300d0080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 -- 172.21.15.37:0/1599515611 >> 172.21.15.37:0/1599515611 conn(0x7f4130009fb0 msgr2=0x7f41300d1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 -- 172.21.15.37:0/1599515611 shutdown_connections 2024-04-26T20:21:57.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.977+0000 7f413d395640 1 -- 172.21.15.37:0/1599515611 wait complete. 2024-04-26T20:21:57.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 -- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feb7006c940 msgr2=0x7feb7006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feb7006c940 0x7feb7006ee00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb84033000 tx=0x7feb84007000 comp rx=0 tx=0).stop 2024-04-26T20:21:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 -- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb9015fca0 msgr2=0x7feb90160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb9015fca0 0x7feb90160120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb800118d0 tx=0x7feb80011d90 comp rx=0 tx=0).stop 2024-04-26T20:21:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 -- 172.21.15.37:0/2387206901 shutdown_connections 2024-04-26T20:21:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feb7006c940 0x7feb7006ee00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb9015d460 0x7feb9015d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb9015fca0 0x7feb90160120 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 --2- 172.21.15.37:0/2387206901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb780039b0 0x7feb9015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 -- 172.21.15.37:0/2387206901 >> 172.21.15.37:0/2387206901 conn(0x7feb9006d350 msgr2=0x7feb9006d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 -- 172.21.15.37:0/2387206901 shutdown_connections 2024-04-26T20:21:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.978+0000 7feb8f59e640 1 -- 172.21.15.37:0/2387206901 wait complete. 2024-04-26T20:21:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.979+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1505082847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ffb881529e0 con 0x7ffb880cd7b0 2024-04-26T20:21:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.980+0000 7ffb7e7fc640 1 -- 172.21.15.37:0/1505082847 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffb7805df90 con 0x7ffb880cd7b0 2024-04-26T20:21:57.983 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.980+0000 7ff7913c9640 1 -- 172.21.15.37:0/2993108301 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff78c0725c0 con 0x7ff78c0b6610 2024-04-26T20:21:57.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.981+0000 7ff7737fe640 1 -- 172.21.15.37:0/2993108301 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff77c05a4c0 con 0x7ff78c0b6610 2024-04-26T20:21:57.984 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:57.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.983+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb6c06c920 msgr2=0x7ffb6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.983+0000 7ffb8eb82640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb6c06c920 0x7ffb6c06ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffb80034000 tx=0x7ffb8002f000 comp rx=0 tx=0).stop 2024-04-26T20:21:57.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1505082847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb880cd7b0 msgr2=0x7ffb88152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ffb8eb82640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb880cd7b0 0x7ffb88152c20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800d800 tx=0x7ffb7800dcc0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1505082847 shutdown_connections 2024-04-26T20:21:57.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ffb8eb82640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb6c06c920 0x7ffb6c06ede0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ffb8eb82640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb88159a70 0x7ffb88072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ffb8eb82640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb880cd7b0 0x7ffb88152c20 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800d800 tx=0x7ffb7800dcc0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ffb8eb82640 1 --2- 172.21.15.37:0/1505082847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb880afae0 0x7ffb8814f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1505082847 >> 172.21.15.37:0/1505082847 conn(0x7ffb880d0ca0 msgr2=0x7ffb8806f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1505082847 shutdown_connections 2024-04-26T20:21:57.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ffb8eb82640 1 -- 172.21.15.37:0/1505082847 wait complete. 2024-04-26T20:21:57.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ff7913c9640 1 -- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff77806c8f0 msgr2=0x7ff77806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ff7913c9640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff77806c8f0 0x7ff77806edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff784037000 tx=0x7ff784032000 comp rx=0 tx=0).stop 2024-04-26T20:21:57.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ff7913c9640 1 -- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff78c0b6610 msgr2=0x7ff78c1f9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ff7913c9640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff78c0b6610 0x7ff78c1f9ec0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00c3e0 tx=0x7ff77c00c8a0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ff7913c9640 1 -- 172.21.15.37:0/2993108301 shutdown_connections 2024-04-26T20:21:57.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ff7913c9640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff77806c8f0 0x7ff77806edb0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff784037000 tx=0x7ff784032000 comp rx=0 tx=0).stop 2024-04-26T20:21:57.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ff7913c9640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff78c158ed0 0x7ff78c0af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ff7913c9640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff78c0b6610 0x7ff78c1f9ec0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ff7913c9640 1 --2- 172.21.15.37:0/2993108301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff78c073f80 0x7ff78c158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.984+0000 7ff7913c9640 1 -- 172.21.15.37:0/2993108301 >> 172.21.15.37:0/2993108301 conn(0x7ff78c0d0d50 msgr2=0x7ff78c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.985+0000 7ff7913c9640 1 -- 172.21.15.37:0/2993108301 shutdown_connections 2024-04-26T20:21:57.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.985+0000 7ff7913c9640 1 -- 172.21.15.37:0/2993108301 wait complete. 2024-04-26T20:21:57.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.988+0000 7f062d031640 1 -- 172.21.15.37:0/2217723383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0620005390 con 0x7f0620006830 2024-04-26T20:21:57.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.989+0000 7f0624ff9640 1 -- 172.21.15.37:0/2217723383 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f061c07abb0 con 0x7f0620006830 2024-04-26T20:21:57.990 INFO:teuthology.orchestra.run.smithi037.stdout:55834574855 2024-04-26T20:21:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.992+0000 7f062d031640 1 -- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f060c06c8f0 msgr2=0x7f060c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.992+0000 7f062d031640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f060c06c8f0 0x7f060c06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0628065980 tx=0x7f062806a160 comp rx=0 tx=0).stop 2024-04-26T20:21:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.992+0000 7f062d031640 1 -- 172.21.15.37:0/2217723383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0620006830 msgr2=0x7f06200e9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.993+0000 7f062d031640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0620006830 0x7f06200e9e90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f061c031000 tx=0x7f061c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.993+0000 7f062d031640 1 -- 172.21.15.37:0/2217723383 shutdown_connections 2024-04-26T20:21:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.993+0000 7f062d031640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0620048040 0x7f06200ea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.993+0000 7f062d031640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0620006830 0x7f06200e9e90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.993+0000 7f062d031640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f060c06c8f0 0x7f060c06edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.993+0000 7f062d031640 1 --2- 172.21.15.37:0/2217723383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0620005e90 0x7f06200d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.993+0000 7f062d031640 1 -- 172.21.15.37:0/2217723383 >> 172.21.15.37:0/2217723383 conn(0x7f0620009fb0 msgr2=0x7f0620037b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.993+0000 7f062d031640 1 -- 172.21.15.37:0/2217723383 shutdown_connections 2024-04-26T20:21:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:57.993+0000 7f062d031640 1 -- 172.21.15.37:0/2217723383 wait complete. 2024-04-26T20:21:57.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-04-26T20:21:57.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-26T20:21:57.998 DEBUG:teuthology.parallel:result is None 2024-04-26T20:21:57.998 DEBUG:teuthology.parallel:result is None 2024-04-26T20:21:57.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-26T20:21:57.998 DEBUG:teuthology.parallel:result is None 2024-04-26T20:21:58.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-26T20:21:58.003 DEBUG:teuthology.parallel:result is None 2024-04-26T20:21:58.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-26T20:21:58.004 DEBUG:teuthology.parallel:result is None 2024-04-26T20:21:58.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-04-26T20:21:58.008 DEBUG:teuthology.parallel:result is None 2024-04-26T20:21:58.928 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T20:21:59.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.041+0000 7f47019b4640 1 Processor -- start 2024-04-26T20:21:59.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.041+0000 7f47019b4640 1 -- start start 2024-04-26T20:21:59.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.042+0000 7f47019b4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46fc0d2180 0x7f46fc0d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.042+0000 7f47019b4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46fc0d3c80 0x7f46fc0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.042+0000 7f47019b4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46fc0d4090 con 0x7f46fc15f400 2024-04-26T20:21:59.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.042+0000 7f47019b4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f46fc0d1290 con 0x7f46fc0d2180 2024-04-26T20:21:59.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.042+0000 7f47019b4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc0d3220 con 0x7f46fc0d3c80 2024-04-26T20:21:59.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.042+0000 7f46f27fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46fc0d2180 0x7f46fc0d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.042+0000 7f46faffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f46fc15f400 0x7f46fc0d0b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44386/0 (socket says 172.21.15.37:44386) 2024-04-26T20:21:59.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.042+0000 7f46f27fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46fc0d2180 0x7f46fc0d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35206/0 (socket says 172.21.15.37:35206) 2024-04-26T20:21:59.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.042+0000 7f46faffd640 1 -- 172.21.15.37:0/413121696 learned_addr learned my addr 172.21.15.37:0/413121696 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:21:59.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.042+0000 7f46fb7fe640 1 --2- 172.21.15.37:0/413121696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46fc0d3c80 0x7f46fc0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.043+0000 7f46f27fc640 1 -- 172.21.15.37:0/413121696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46fc0d3c80 msgr2=0x7f46fc0b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:59.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.043+0000 7f46f27fc640 1 --2- 172.21.15.37:0/413121696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46fc0d3c80 0x7f46fc0b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.043+0000 7f46f27fc640 1 -- 172.21.15.37:0/413121696 >> v1:172.21.15.37:6789/0 conn(0x7f46fc15f400 legacy=0x7f46fc0d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.043+0000 7f46fa7fc640 1 -- 172.21.15.37:0/413121696 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1049634333 0 0) 0x7f46ec001110 con 0x7f46fc15f400 2024-04-26T20:21:59.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.043+0000 7f46f27fc640 1 -- 172.21.15.37:0/413121696 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc0d5ef0 con 0x7f46fc0d2180 2024-04-26T20:21:59.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.043+0000 7f46fb7fe640 1 --2- 172.21.15.37:0/413121696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46fc0d3c80 0x7f46fc0b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:59.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.043+0000 7f46f27fc640 1 --2- 172.21.15.37:0/413121696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46fc0d2180 0x7f46fc0d3740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f46e40097a0 tx=0x7f46e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48cc1b177d2015 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:59.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.044+0000 7f46fa7fc640 1 -- 172.21.15.37:0/413121696 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4037070 con 0x7f46fc0d2180 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.044+0000 7f46fa7fc640 1 -- 172.21.15.37:0/413121696 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e4032440 con 0x7f46fc0d2180 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.044+0000 7f46fa7fc640 1 -- 172.21.15.37:0/413121696 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4032810 con 0x7f46fc0d2180 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.044+0000 7f47019b4640 1 -- 172.21.15.37:0/413121696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46fc0d2180 msgr2=0x7f46fc0d3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.044+0000 7f47019b4640 1 --2- 172.21.15.37:0/413121696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46fc0d2180 0x7f46fc0d3740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f46e40097a0 tx=0x7f46e402a180 comp rx=0 tx=0).stop 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.044+0000 7f47019b4640 1 -- 172.21.15.37:0/413121696 shutdown_connections 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.044+0000 7f47019b4640 1 --2- 172.21.15.37:0/413121696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46fc0d3c80 0x7f46fc0b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.044+0000 7f47019b4640 1 --2- 172.21.15.37:0/413121696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46fc0d2180 0x7f46fc0d3740 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.044+0000 7f47019b4640 1 -- 172.21.15.37:0/413121696 >> 172.21.15.37:0/413121696 conn(0x7f46fc09e510 msgr2=0x7f46fc0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.044+0000 7f47019b4640 1 -- 172.21.15.37:0/413121696 shutdown_connections 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.044+0000 7f47019b4640 1 -- 172.21.15.37:0/413121696 wait complete. 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.045+0000 7f47019b4640 1 Processor -- start 2024-04-26T20:21:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.046+0000 7f47019b4640 1 -- start start 2024-04-26T20:21:59.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.046+0000 7f47019b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46fc0d2180 0x7f46fc0cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.046+0000 7f47019b4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46fc0d3c80 0x7f46fc1f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.046+0000 7f47019b4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46fc15f400 0x7f46fc161410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.046+0000 7f47019b4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc0d0660 con 0x7f46fc0d2180 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.046+0000 7f47019b4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc0d0380 con 0x7f46fc15f400 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.046+0000 7f47019b4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc0d25f0 con 0x7f46fc0d3c80 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.046+0000 7f46faffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46fc0d2180 0x7f46fc0cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.046+0000 7f46faffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46fc0d2180 0x7f46fc0cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35220/0 (socket says 172.21.15.37:35220) 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.046+0000 7f46faffd640 1 -- 172.21.15.37:0/3507622617 learned_addr learned my addr 172.21.15.37:0/3507622617 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46f27fc640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46fc0d3c80 0x7f46fc1f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46fb7fe640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46fc15f400 0x7f46fc161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46faffd640 1 -- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46fc0d3c80 msgr2=0x7f46fc1f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46faffd640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46fc0d3c80 0x7f46fc1f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46faffd640 1 -- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46fc15f400 msgr2=0x7f46fc161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46faffd640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46fc15f400 0x7f46fc161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46faffd640 1 -- 172.21.15.37:0/3507622617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc15f7e0 con 0x7f46fc0d2180 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46fb7fe640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46fc15f400 0x7f46fc161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46faffd640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46fc0d2180 0x7f46fc0cf3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c780 tx=0x7f46ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46f8ff9640 1 -- 172.21.15.37:0/3507622617 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec018070 con 0x7f46fc0d2180 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46f8ff9640 1 -- 172.21.15.37:0/3507622617 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec0072d0 con 0x7f46fc0d2180 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f47019b4640 1 -- 172.21.15.37:0/3507622617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc1f6020 con 0x7f46fc0d2180 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f46f8ff9640 1 -- 172.21.15.37:0/3507622617 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0047e0 con 0x7f46fc0d2180 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.047+0000 7f47019b4640 1 -- 172.21.15.37:0/3507622617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc15fd60 con 0x7f46fc0d2180 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.048+0000 7f47019b4640 1 -- 172.21.15.37:0/3507622617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46fc0693e0 con 0x7f46fc0d2180 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.049+0000 7f46f8ff9640 1 -- 172.21.15.37:0/3507622617 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46ec01b070 con 0x7f46fc0d2180 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.049+0000 7f46f8ff9640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f46dc06c8f0 0x7f46dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.049+0000 7f46f8ff9640 1 -- 172.21.15.37:0/3507622617 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f46ec08adc0 con 0x7f46fc0d2180 2024-04-26T20:21:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.049+0000 7f46f27fc640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f46dc06c8f0 0x7f46dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.050+0000 7f46f27fc640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f46dc06c8f0 0x7f46dc06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f46e4034000 tx=0x7f46e402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:59.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.054+0000 7f46f8ff9640 1 -- 172.21.15.37:0/3507622617 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46fc0d4090 con 0x7f46fc0d2180 2024-04-26T20:21:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.239+0000 7f47019b4640 1 -- 172.21.15.37:0/3507622617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f46fc0cb910 con 0x7f46fc0d2180 2024-04-26T20:21:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.239+0000 7f46f8ff9640 1 -- 172.21.15.37:0/3507622617 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f46ec058ba0 con 0x7f46fc0d2180 2024-04-26T20:21:59.241 INFO:teuthology.orchestra.run.smithi037.stdout:55834574856 2024-04-26T20:21:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 -- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f46dc06c8f0 msgr2=0x7f46dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f46dc06c8f0 0x7f46dc06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f46e4034000 tx=0x7f46e402e000 comp rx=0 tx=0).stop 2024-04-26T20:21:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 -- 172.21.15.37:0/3507622617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46fc0d2180 msgr2=0x7f46fc0cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46fc0d2180 0x7f46fc0cf3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c780 tx=0x7f46ec00cc40 comp rx=0 tx=0).stop 2024-04-26T20:21:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 -- 172.21.15.37:0/3507622617 shutdown_connections 2024-04-26T20:21:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46fc15f400 0x7f46fc161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f46dc06c8f0 0x7f46dc06edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46fc0d3c80 0x7f46fc1f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 --2- 172.21.15.37:0/3507622617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46fc0d2180 0x7f46fc0cf3e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 -- 172.21.15.37:0/3507622617 >> 172.21.15.37:0/3507622617 conn(0x7f46fc09e510 msgr2=0x7f46fc06d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 -- 172.21.15.37:0/3507622617 shutdown_connections 2024-04-26T20:21:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.241+0000 7f47019b4640 1 -- 172.21.15.37:0/3507622617 wait complete. 2024-04-26T20:21:59.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-04-26T20:21:59.253 DEBUG:teuthology.parallel:result is None 2024-04-26T20:21:59.253 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T20:21:59.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:21:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.354+0000 7fd6dc8ea640 1 Processor -- start 2024-04-26T20:21:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.354+0000 7fd6dc8ea640 1 -- start start 2024-04-26T20:21:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.354+0000 7fd6dc8ea640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd6d8162e40 0x7fd6d80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.354+0000 7fd6dc8ea640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd6d80cff40 0x7fd6d80ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.354+0000 7fd6dc8ea640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d80b1d70 con 0x7fd6d80b2050 2024-04-26T20:21:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.354+0000 7fd6dc8ea640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d8163220 con 0x7fd6d80cff40 2024-04-26T20:21:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.354+0000 7fd6dc8ea640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d80d0eb0 con 0x7fd6d8162e40 2024-04-26T20:21:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.355+0000 7fd6d6d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd6d80cff40 0x7fd6d80ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.355+0000 7fd6d6d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd6d80cff40 0x7fd6d80ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35230/0 (socket says 172.21.15.37:35230) 2024-04-26T20:21:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.355+0000 7fd6d6d9d640 1 -- 172.21.15.37:0/146778660 learned_addr learned my addr 172.21.15.37:0/146778660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.355+0000 7fd6d659c640 1 --2- 172.21.15.37:0/146778660 >> v2:172.21.15.134:3300/0 conn(0x7fd6d8162e40 0x7fd6d80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.355+0000 7fd6d6d9d640 1 -- 172.21.15.37:0/146778660 >> v2:172.21.15.134:3300/0 conn(0x7fd6d8162e40 msgr2=0x7fd6d80d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.355+0000 7fd6d6d9d640 1 --2- 172.21.15.37:0/146778660 >> v2:172.21.15.134:3300/0 conn(0x7fd6d8162e40 0x7fd6d80d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.355+0000 7fd6d6d9d640 1 -- 172.21.15.37:0/146778660 >> v1:172.21.15.134:6789/0 conn(0x7fd6d80b2050 legacy=0x7fd6d80cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.355+0000 7fd6d6d9d640 1 -- 172.21.15.37:0/146778660 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d80d1960 con 0x7fd6d80cff40 2024-04-26T20:21:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.355+0000 7fd6d6d9d640 1 --2- 172.21.15.37:0/146778660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d80cff40 0x7fd6d80ce010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8075e20 tx=0x7fd6d002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7abefb63360c9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.356+0000 7fd6d5d9b640 1 -- 172.21.15.37:0/146778660 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0036070 con 0x7fd6d80cff40 2024-04-26T20:21:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.356+0000 7fd6d5d9b640 1 -- 172.21.15.37:0/146778660 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d0031600 con 0x7fd6d80cff40 2024-04-26T20:21:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.356+0000 7fd6d5d9b640 1 -- 172.21.15.37:0/146778660 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00319d0 con 0x7fd6d80cff40 2024-04-26T20:21:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.356+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/146778660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d80cff40 msgr2=0x7fd6d80ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.356+0000 7fd6dc8ea640 1 --2- 172.21.15.37:0/146778660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d80cff40 0x7fd6d80ce010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8075e20 tx=0x7fd6d002a660 comp rx=0 tx=0).stop 2024-04-26T20:21:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.356+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/146778660 shutdown_connections 2024-04-26T20:21:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.356+0000 7fd6dc8ea640 1 --2- 172.21.15.37:0/146778660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d80cff40 0x7fd6d80ce010 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.356+0000 7fd6dc8ea640 1 --2- 172.21.15.37:0/146778660 >> v2:172.21.15.134:3300/0 conn(0x7fd6d8162e40 0x7fd6d80d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.356+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/146778660 >> 172.21.15.37:0/146778660 conn(0x7fd6d809ba50 msgr2=0x7fd6d809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:59.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.356+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/146778660 shutdown_connections 2024-04-26T20:21:59.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.356+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/146778660 wait complete. 2024-04-26T20:21:59.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6dc8ea640 1 Processor -- start 2024-04-26T20:21:59.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6dc8ea640 1 -- start start 2024-04-26T20:21:59.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6dc8ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6d80b2050 0x7fd6d80716c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6dc8ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6d80cff40 0x7fd6d81f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6dc8ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d8162e40 0x7fd6d81f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6dc8ea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d80b24e0 con 0x7fd6d8162e40 2024-04-26T20:21:59.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6dc8ea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d80d1f20 con 0x7fd6d80cff40 2024-04-26T20:21:59.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6dc8ea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d806ea40 con 0x7fd6d80b2050 2024-04-26T20:21:59.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d8162e40 0x7fd6d81f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d8162e40 0x7fd6d81f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35238/0 (socket says 172.21.15.37:35238) 2024-04-26T20:21:59.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6d6d9d640 1 -- 172.21.15.37:0/363879502 learned_addr learned my addr 172.21.15.37:0/363879502 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:59.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6d659c640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6d80b2050 0x7fd6d80716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6cffff640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6d80cff40 0x7fd6d81f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6d6d9d640 1 -- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6d80b2050 msgr2=0x7fd6d80716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6d6d9d640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6d80b2050 0x7fd6d80716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6d6d9d640 1 -- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6d80cff40 msgr2=0x7fd6d81f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6d6d9d640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6d80cff40 0x7fd6d81f45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6d6d9d640 1 -- 172.21.15.37:0/363879502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d815bed0 con 0x7fd6d8162e40 2024-04-26T20:21:59.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6d6d9d640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d8162e40 0x7fd6d81f4b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0034040 tx=0x7fd6d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:59.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6ceffd640 1 -- 172.21.15.37:0/363879502 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0036050 con 0x7fd6d8162e40 2024-04-26T20:21:59.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6ceffd640 1 -- 172.21.15.37:0/363879502 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d0042070 con 0x7fd6d8162e40 2024-04-26T20:21:59.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/363879502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d815c1b0 con 0x7fd6d8162e40 2024-04-26T20:21:59.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6ceffd640 1 -- 172.21.15.37:0/363879502 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0009510 con 0x7fd6d8162e40 2024-04-26T20:21:59.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.359+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/363879502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d815e7a0 con 0x7fd6d8162e40 2024-04-26T20:21:59.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.360+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/363879502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d806a360 con 0x7fd6d8162e40 2024-04-26T20:21:59.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.361+0000 7fd6ceffd640 1 -- 172.21.15.37:0/363879502 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6d0007bd0 con 0x7fd6d8162e40 2024-04-26T20:21:59.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.361+0000 7fd6ceffd640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd6b806c8f0 0x7fd6b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.361+0000 7fd6ceffd640 1 -- 172.21.15.37:0/363879502 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd6d00ac830 con 0x7fd6d8162e40 2024-04-26T20:21:59.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.361+0000 7fd6d659c640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd6b806c8f0 0x7fd6b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.362+0000 7fd6d659c640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd6b806c8f0 0x7fd6b806edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80079a0 tx=0x7fd6c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:59.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.366+0000 7fd6ceffd640 1 -- 172.21.15.37:0/363879502 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6d007a6a0 con 0x7fd6d8162e40 2024-04-26T20:21:59.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.548+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/363879502 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6d806efe0 con 0x7fd6b806c8f0 2024-04-26T20:21:59.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.555+0000 7fd6ceffd640 1 -- 172.21.15.37:0/363879502 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fd6d806efe0 con 0x7fd6b806c8f0 2024-04-26T20:21:59.558 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:59.559 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:21:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd6b806c8f0 msgr2=0x7fd6b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd6b806c8f0 0x7fd6b806edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80079a0 tx=0x7fd6c8008040 comp rx=0 tx=0).stop 2024-04-26T20:21:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/363879502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d8162e40 msgr2=0x7fd6d81f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d8162e40 0x7fd6d81f4b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0034040 tx=0x7fd6d0004a40 comp rx=0 tx=0).stop 2024-04-26T20:21:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/363879502 shutdown_connections 2024-04-26T20:21:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d8162e40 0x7fd6d81f4b20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6d80cff40 0x7fd6d81f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd6b806c8f0 0x7fd6b806edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 --2- 172.21.15.37:0/363879502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6d80b2050 0x7fd6d80716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/363879502 >> 172.21.15.37:0/363879502 conn(0x7fd6d809ba50 msgr2=0x7fd6d809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/363879502 shutdown_connections 2024-04-26T20:21:59.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.558+0000 7fd6dc8ea640 1 -- 172.21.15.37:0/363879502 wait complete. 2024-04-26T20:21:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-26T20:21:59.336643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.465143"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317460+0000","last_change":"2024-04-26T20:21:49.899185+0000","last_active":"2024-04-26T20:21:51.317460+0000","last_peered":"2024-04-26T20:21:51.317460+0000","last_clean":"2024-04-26T20:21:51.317460+0000","last_became_active":"2024-04-26T20:21:49.898818+0000","last_became_peered":"2024-04-26T20:21:49.898818+0000","last_unstale":"2024-04-26T20:21:51.317460+0000","last_undegraded":"2024-04-26T20:21:51.317460+0000","last_fullsized":"2024-04-26T20:21:51.317460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:13.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.889053+0000","last_change":"2024-04-26T20:21:49.899484+0000","last_active":"2024-04-26T20:21:50.889053+0000","last_peered":"2024-04-26T20:21:50.889053+0000","last_clean":"2024-04-26T20:21:50.889053+0000","last_became_active":"2024-04-26T20:21:49.899091+0000","last_became_peered":"2024-04-26T20:21:49.899091+0000","last_unstale":"2024-04-26T20:21:50.889053+0000","last_undegraded":"2024-04-26T20:21:50.889053+0000","last_fullsized":"2024-04-26T20:21:50.889053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.887979+0000","last_change":"2024-04-26T20:21:49.881814+0000","last_active":"2024-04-26T20:21:50.887979+0000","last_peered":"2024-04-26T20:21:50.887979+0000","last_clean":"2024-04-26T20:21:50.887979+0000","last_became_active":"2024-04-26T20:21:49.881507+0000","last_became_peered":"2024-04-26T20:21:49.881507+0000","last_unstale":"2024-04-26T20:21:50.887979+0000","last_undegraded":"2024-04-26T20:21:50.887979+0000","last_fullsized":"2024-04-26T20:21:50.887979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:30.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318197+0000","last_change":"2024-04-26T20:21:49.896399+0000","last_active":"2024-04-26T20:21:51.318197+0000","last_peered":"2024-04-26T20:21:51.318197+0000","last_clean":"2024-04-26T20:21:51.318197+0000","last_became_active":"2024-04-26T20:21:49.896136+0000","last_became_peered":"2024-04-26T20:21:49.896136+0000","last_unstale":"2024-04-26T20:21:51.318197+0000","last_undegraded":"2024-04-26T20:21:51.318197+0000","last_fullsized":"2024-04-26T20:21:51.318197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.737636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317925+0000","last_change":"2024-04-26T20:21:49.898821+0000","last_active":"2024-04-26T20:21:51.317925+0000","last_peered":"2024-04-26T20:21:51.317925+0000","last_clean":"2024-04-26T20:21:51.317925+0000","last_became_active":"2024-04-26T20:21:49.898554+0000","last_became_peered":"2024-04-26T20:21:49.898554+0000","last_unstale":"2024-04-26T20:21:51.317925+0000","last_undegraded":"2024-04-26T20:21:51.317925+0000","last_fullsized":"2024-04-26T20:21:51.317925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:52.618628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316822+0000","last_change":"2024-04-26T20:21:49.878509+0000","last_active":"2024-04-26T20:21:51.316822+0000","last_peered":"2024-04-26T20:21:51.316822+0000","last_clean":"2024-04-26T20:21:51.316822+0000","last_became_active":"2024-04-26T20:21:49.878237+0000","last_became_peered":"2024-04-26T20:21:49.878237+0000","last_unstale":"2024-04-26T20:21:51.316822+0000","last_undegraded":"2024-04-26T20:21:51.316822+0000","last_fullsized":"2024-04-26T20:21:51.316822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316684+0000","last_change":"2024-04-26T20:21:49.898211+0000","last_active":"2024-04-26T20:21:51.316684+0000","last_peered":"2024-04-26T20:21:51.316684+0000","last_clean":"2024-04-26T20:21:51.316684+0000","last_became_active":"2024-04-26T20:21:49.897952+0000","last_became_peered":"2024-04-26T20:21:49.897952+0000","last_unstale":"2024-04-26T20:21:51.316684+0000","last_undegraded":"2024-04-26T20:21:51.316684+0000","last_fullsized":"2024-04-26T20:21:51.316684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316222+0000","last_change":"2024-04-26T20:21:49.892897+0000","last_active":"2024-04-26T20:21:51.316222+0000","last_peered":"2024-04-26T20:21:51.316222+0000","last_clean":"2024-04-26T20:21:51.316222+0000","last_became_active":"2024-04-26T20:21:49.890640+0000","last_became_peered":"2024-04-26T20:21:49.890640+0000","last_unstale":"2024-04-26T20:21:51.316222+0000","last_undegraded":"2024-04-26T20:21:51.316222+0000","last_fullsized":"2024-04-26T20:21:51.316222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:42.131574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317597+0000","last_change":"2024-04-26T20:21:49.901710+0000","last_active":"2024-04-26T20:21:51.317597+0000","last_peered":"2024-04-26T20:21:51.317597+0000","last_clean":"2024-04-26T20:21:51.317597+0000","last_became_active":"2024-04-26T20:21:49.901363+0000","last_became_peered":"2024-04-26T20:21:49.901363+0000","last_unstale":"2024-04-26T20:21:51.317597+0000","last_undegraded":"2024-04-26T20:21:51.317597+0000","last_fullsized":"2024-04-26T20:21:51.317597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:50.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891437+0000","last_change":"2024-04-26T20:21:49.900636+0000","last_active":"2024-04-26T20:21:50.891437+0000","last_peered":"2024-04-26T20:21:50.891437+0000","last_clean":"2024-04-26T20:21:50.891437+0000","last_became_active":"2024-04-26T20:21:49.900073+0000","last_became_peered":"2024-04-26T20:21:49.900073+0000","last_unstale":"2024-04-26T20:21:50.891437+0000","last_undegraded":"2024-04-26T20:21:50.891437+0000","last_fullsized":"2024-04-26T20:21:50.891437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891162+0000","last_change":"2024-04-26T20:21:49.892413+0000","last_active":"2024-04-26T20:21:50.891162+0000","last_peered":"2024-04-26T20:21:50.891162+0000","last_clean":"2024-04-26T20:21:50.891162+0000","last_became_active":"2024-04-26T20:21:49.892106+0000","last_became_peered":"2024-04-26T20:21:49.892106+0000","last_unstale":"2024-04-26T20:21:50.891162+0000","last_undegraded":"2024-04-26T20:21:50.891162+0000","last_fullsized":"2024-04-26T20:21:50.891162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317031+0000","last_change":"2024-04-26T20:21:49.894073+0000","last_active":"2024-04-26T20:21:51.317031+0000","last_peered":"2024-04-26T20:21:51.317031+0000","last_clean":"2024-04-26T20:21:51.317031+0000","last_became_active":"2024-04-26T20:21:49.893726+0000","last_became_peered":"2024-04-26T20:21:49.893726+0000","last_unstale":"2024-04-26T20:21:51.317031+0000","last_undegraded":"2024-04-26T20:21:51.317031+0000","last_fullsized":"2024-04-26T20:21:51.317031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:00.300212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316742+0000","last_change":"2024-04-26T20:21:49.895923+0000","last_active":"2024-04-26T20:21:51.316742+0000","last_peered":"2024-04-26T20:21:51.316742+0000","last_clean":"2024-04-26T20:21:51.316742+0000","last_became_active":"2024-04-26T20:21:49.895261+0000","last_became_peered":"2024-04-26T20:21:49.895261+0000","last_unstale":"2024-04-26T20:21:51.316742+0000","last_undegraded":"2024-04-26T20:21:51.316742+0000","last_fullsized":"2024-04-26T20:21:51.316742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.887942+0000","last_change":"2024-04-26T20:21:49.904360+0000","last_active":"2024-04-26T20:21:50.887942+0000","last_peered":"2024-04-26T20:21:50.887942+0000","last_clean":"2024-04-26T20:21:50.887942+0000","last_became_active":"2024-04-26T20:21:49.899936+0000","last_became_peered":"2024-04-26T20:21:49.899936+0000","last_unstale":"2024-04-26T20:21:50.887942+0000","last_undegraded":"2024-04-26T20:21:50.887942+0000","last_fullsized":"2024-04-26T20:21:50.887942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:55.987722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888860+0000","last_change":"2024-04-26T20:21:49.893445+0000","last_active":"2024-04-26T20:21:50.888860+0000","last_peered":"2024-04-26T20:21:50.888860+0000","last_clean":"2024-04-26T20:21:50.888860+0000","last_became_active":"2024-04-26T20:21:49.892704+0000","last_became_peered":"2024-04-26T20:21:49.892704+0000","last_unstale":"2024-04-26T20:21:50.888860+0000","last_undegraded":"2024-04-26T20:21:50.888860+0000","last_fullsized":"2024-04-26T20:21:50.888860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316915+0000","last_change":"2024-04-26T20:21:49.883172+0000","last_active":"2024-04-26T20:21:51.316915+0000","last_peered":"2024-04-26T20:21:51.316915+0000","last_clean":"2024-04-26T20:21:51.316915+0000","last_became_active":"2024-04-26T20:21:49.882916+0000","last_became_peered":"2024-04-26T20:21:49.882916+0000","last_unstale":"2024-04-26T20:21:51.316915+0000","last_undegraded":"2024-04-26T20:21:51.316915+0000","last_fullsized":"2024-04-26T20:21:51.316915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317389+0000","last_change":"2024-04-26T20:21:49.889742+0000","last_active":"2024-04-26T20:21:51.317389+0000","last_peered":"2024-04-26T20:21:51.317389+0000","last_clean":"2024-04-26T20:21:51.317389+0000","last_became_active":"2024-04-26T20:21:49.889417+0000","last_became_peered":"2024-04-26T20:21:49.889417+0000","last_unstale":"2024-04-26T20:21:51.317389+0000","last_undegraded":"2024-04-26T20:21:51.317389+0000","last_fullsized":"2024-04-26T20:21:51.317389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:53.734129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890942+0000","last_change":"2024-04-26T20:21:49.894387+0000","last_active":"2024-04-26T20:21:50.890942+0000","last_peered":"2024-04-26T20:21:50.890942+0000","last_clean":"2024-04-26T20:21:50.890942+0000","last_became_active":"2024-04-26T20:21:49.893464+0000","last_became_peered":"2024-04-26T20:21:49.893464+0000","last_unstale":"2024-04-26T20:21:50.890942+0000","last_undegraded":"2024-04-26T20:21:50.890942+0000","last_fullsized":"2024-04-26T20:21:50.890942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:26.689749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317646+0000","last_change":"2024-04-26T20:21:49.892977+0000","last_active":"2024-04-26T20:21:51.317646+0000","last_peered":"2024-04-26T20:21:51.317646+0000","last_clean":"2024-04-26T20:21:51.317646+0000","last_became_active":"2024-04-26T20:21:49.892709+0000","last_became_peered":"2024-04-26T20:21:49.892709+0000","last_unstale":"2024-04-26T20:21:51.317646+0000","last_undegraded":"2024-04-26T20:21:51.317646+0000","last_fullsized":"2024-04-26T20:21:51.317646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:44.689592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891273+0000","last_change":"2024-04-26T20:21:49.892717+0000","last_active":"2024-04-26T20:21:50.891273+0000","last_peered":"2024-04-26T20:21:50.891273+0000","last_clean":"2024-04-26T20:21:50.891273+0000","last_became_active":"2024-04-26T20:21:49.892029+0000","last_became_peered":"2024-04-26T20:21:49.892029+0000","last_unstale":"2024-04-26T20:21:50.891273+0000","last_undegraded":"2024-04-26T20:21:50.891273+0000","last_fullsized":"2024-04-26T20:21:50.891273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888152+0000","last_change":"2024-04-26T20:21:49.891937+0000","last_active":"2024-04-26T20:21:50.888152+0000","last_peered":"2024-04-26T20:21:50.888152+0000","last_clean":"2024-04-26T20:21:50.888152+0000","last_became_active":"2024-04-26T20:21:49.891631+0000","last_became_peered":"2024-04-26T20:21:49.891631+0000","last_unstale":"2024-04-26T20:21:50.888152+0000","last_undegraded":"2024-04-26T20:21:50.888152+0000","last_fullsized":"2024-04-26T20:21:50.888152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318190+0000","last_change":"2024-04-26T20:21:49.890285+0000","last_active":"2024-04-26T20:21:51.318190+0000","last_peered":"2024-04-26T20:21:51.318190+0000","last_clean":"2024-04-26T20:21:51.318190+0000","last_became_active":"2024-04-26T20:21:49.888800+0000","last_became_peered":"2024-04-26T20:21:49.888800+0000","last_unstale":"2024-04-26T20:21:51.318190+0000","last_undegraded":"2024-04-26T20:21:51.318190+0000","last_fullsized":"2024-04-26T20:21:51.318190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:32.476892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316465+0000","last_change":"2024-04-26T20:21:49.890345+0000","last_active":"2024-04-26T20:21:51.316465+0000","last_peered":"2024-04-26T20:21:51.316465+0000","last_clean":"2024-04-26T20:21:51.316465+0000","last_became_active":"2024-04-26T20:21:49.889999+0000","last_became_peered":"2024-04-26T20:21:49.889999+0000","last_unstale":"2024-04-26T20:21:51.316465+0000","last_undegraded":"2024-04-26T20:21:51.316465+0000","last_fullsized":"2024-04-26T20:21:51.316465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317064+0000","last_change":"2024-04-26T20:21:49.897497+0000","last_active":"2024-04-26T20:21:51.317064+0000","last_peered":"2024-04-26T20:21:51.317064+0000","last_clean":"2024-04-26T20:21:51.317064+0000","last_became_active":"2024-04-26T20:21:49.896968+0000","last_became_peered":"2024-04-26T20:21:49.896968+0000","last_unstale":"2024-04-26T20:21:51.317064+0000","last_undegraded":"2024-04-26T20:21:51.317064+0000","last_fullsized":"2024-04-26T20:21:51.317064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316953+0000","last_change":"2024-04-26T20:21:49.890916+0000","last_active":"2024-04-26T20:21:51.316953+0000","last_peered":"2024-04-26T20:21:51.316953+0000","last_clean":"2024-04-26T20:21:51.316953+0000","last_became_active":"2024-04-26T20:21:49.890517+0000","last_became_peered":"2024-04-26T20:21:49.890517+0000","last_unstale":"2024-04-26T20:21:51.316953+0000","last_undegraded":"2024-04-26T20:21:51.316953+0000","last_fullsized":"2024-04-26T20:21:51.316953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891447+0000","last_change":"2024-04-26T20:21:49.903237+0000","last_active":"2024-04-26T20:21:50.891447+0000","last_peered":"2024-04-26T20:21:50.891447+0000","last_clean":"2024-04-26T20:21:50.891447+0000","last_became_active":"2024-04-26T20:21:49.902924+0000","last_became_peered":"2024-04-26T20:21:49.902924+0000","last_unstale":"2024-04-26T20:21:50.891447+0000","last_undegraded":"2024-04-26T20:21:50.891447+0000","last_fullsized":"2024-04-26T20:21:50.891447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888809+0000","last_change":"2024-04-26T20:21:49.905100+0000","last_active":"2024-04-26T20:21:50.888809+0000","last_peered":"2024-04-26T20:21:50.888809+0000","last_clean":"2024-04-26T20:21:50.888809+0000","last_became_active":"2024-04-26T20:21:49.904037+0000","last_became_peered":"2024-04-26T20:21:49.904037+0000","last_unstale":"2024-04-26T20:21:50.888809+0000","last_undegraded":"2024-04-26T20:21:50.888809+0000","last_fullsized":"2024-04-26T20:21:50.888809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316677+0000","last_change":"2024-04-26T20:21:49.891939+0000","last_active":"2024-04-26T20:21:51.316677+0000","last_peered":"2024-04-26T20:21:51.316677+0000","last_clean":"2024-04-26T20:21:51.316677+0000","last_became_active":"2024-04-26T20:21:49.891662+0000","last_became_peered":"2024-04-26T20:21:49.891662+0000","last_unstale":"2024-04-26T20:21:51.316677+0000","last_undegraded":"2024-04-26T20:21:51.316677+0000","last_fullsized":"2024-04-26T20:21:51.316677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888299+0000","last_change":"2024-04-26T20:21:49.895730+0000","last_active":"2024-04-26T20:21:50.888299+0000","last_peered":"2024-04-26T20:21:50.888299+0000","last_clean":"2024-04-26T20:21:50.888299+0000","last_became_active":"2024-04-26T20:21:49.895244+0000","last_became_peered":"2024-04-26T20:21:49.895244+0000","last_unstale":"2024-04-26T20:21:50.888299+0000","last_undegraded":"2024-04-26T20:21:50.888299+0000","last_fullsized":"2024-04-26T20:21:50.888299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318438+0000","last_change":"2024-04-26T20:21:49.896367+0000","last_active":"2024-04-26T20:21:51.318438+0000","last_peered":"2024-04-26T20:21:51.318438+0000","last_clean":"2024-04-26T20:21:51.318438+0000","last_became_active":"2024-04-26T20:21:49.895963+0000","last_became_peered":"2024-04-26T20:21:49.895963+0000","last_unstale":"2024-04-26T20:21:51.318438+0000","last_undegraded":"2024-04-26T20:21:51.318438+0000","last_fullsized":"2024-04-26T20:21:51.318438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:57.212796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890871+0000","last_change":"2024-04-26T20:21:49.880643+0000","last_active":"2024-04-26T20:21:50.890871+0000","last_peered":"2024-04-26T20:21:50.890871+0000","last_clean":"2024-04-26T20:21:50.890871+0000","last_became_active":"2024-04-26T20:21:49.880295+0000","last_became_peered":"2024-04-26T20:21:49.880295+0000","last_unstale":"2024-04-26T20:21:50.890871+0000","last_undegraded":"2024-04-26T20:21:50.890871+0000","last_fullsized":"2024-04-26T20:21:50.890871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:40.680121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891215+0000","last_change":"2024-04-26T20:21:49.906308+0000","last_active":"2024-04-26T20:21:50.891215+0000","last_peered":"2024-04-26T20:21:50.891215+0000","last_clean":"2024-04-26T20:21:50.891215+0000","last_became_active":"2024-04-26T20:21:49.906092+0000","last_became_peered":"2024-04-26T20:21:49.906092+0000","last_unstale":"2024-04-26T20:21:50.891215+0000","last_undegraded":"2024-04-26T20:21:50.891215+0000","last_fullsized":"2024-04-26T20:21:50.891215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318011+0000","last_change":"2024-04-26T20:21:49.901086+0000","last_active":"2024-04-26T20:21:51.318011+0000","last_peered":"2024-04-26T20:21:51.318011+0000","last_clean":"2024-04-26T20:21:51.318011+0000","last_became_active":"2024-04-26T20:21:49.900797+0000","last_became_peered":"2024-04-26T20:21:49.900797+0000","last_unstale":"2024-04-26T20:21:51.318011+0000","last_undegraded":"2024-04-26T20:21:51.318011+0000","last_fullsized":"2024-04-26T20:21:51.318011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:26.689749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316705+0000","last_change":"2024-04-26T20:21:48.880856+0000","last_active":"2024-04-26T20:21:51.316705+0000","last_peered":"2024-04-26T20:21:51.316705+0000","last_clean":"2024-04-26T20:21:51.316705+0000","last_became_active":"2024-04-26T20:21:48.880512+0000","last_became_peered":"2024-04-26T20:21:48.880512+0000","last_unstale":"2024-04-26T20:21:51.316705+0000","last_undegraded":"2024-04-26T20:21:51.316705+0000","last_fullsized":"2024-04-26T20:21:51.316705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888381+0000","last_change":"2024-04-26T20:21:49.889888+0000","last_active":"2024-04-26T20:21:50.888381+0000","last_peered":"2024-04-26T20:21:50.888381+0000","last_clean":"2024-04-26T20:21:50.888381+0000","last_became_active":"2024-04-26T20:21:49.889451+0000","last_became_peered":"2024-04-26T20:21:49.889451+0000","last_unstale":"2024-04-26T20:21:50.888381+0000","last_undegraded":"2024-04-26T20:21:50.888381+0000","last_fullsized":"2024-04-26T20:21:50.888381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891210+0000","last_change":"2024-04-26T20:21:48.880648+0000","last_active":"2024-04-26T20:21:50.891210+0000","last_peered":"2024-04-26T20:21:50.891210+0000","last_clean":"2024-04-26T20:21:50.891210+0000","last_became_active":"2024-04-26T20:21:48.880389+0000","last_became_peered":"2024-04-26T20:21:48.880389+0000","last_unstale":"2024-04-26T20:21:50.891210+0000","last_undegraded":"2024-04-26T20:21:50.891210+0000","last_fullsized":"2024-04-26T20:21:50.891210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316415+0000","last_change":"2024-04-26T20:21:48.873199+0000","last_active":"2024-04-26T20:21:51.316415+0000","last_peered":"2024-04-26T20:21:51.316415+0000","last_clean":"2024-04-26T20:21:51.316415+0000","last_became_active":"2024-04-26T20:21:48.872868+0000","last_became_peered":"2024-04-26T20:21:48.872868+0000","last_unstale":"2024-04-26T20:21:51.316415+0000","last_undegraded":"2024-04-26T20:21:51.316415+0000","last_fullsized":"2024-04-26T20:21:51.316415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316315+0000","last_change":"2024-04-26T20:21:49.897450+0000","last_active":"2024-04-26T20:21:51.316315+0000","last_peered":"2024-04-26T20:21:51.316315+0000","last_clean":"2024-04-26T20:21:51.316315+0000","last_became_active":"2024-04-26T20:21:49.897165+0000","last_became_peered":"2024-04-26T20:21:49.897165+0000","last_unstale":"2024-04-26T20:21:51.316315+0000","last_undegraded":"2024-04-26T20:21:51.316315+0000","last_fullsized":"2024-04-26T20:21:51.316315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317170+0000","last_change":"2024-04-26T20:21:48.881023+0000","last_active":"2024-04-26T20:21:51.317170+0000","last_peered":"2024-04-26T20:21:51.317170+0000","last_clean":"2024-04-26T20:21:51.317170+0000","last_became_active":"2024-04-26T20:21:48.880610+0000","last_became_peered":"2024-04-26T20:21:48.880610+0000","last_unstale":"2024-04-26T20:21:51.317170+0000","last_undegraded":"2024-04-26T20:21:51.317170+0000","last_fullsized":"2024-04-26T20:21:51.317170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888715+0000","last_change":"2024-04-26T20:21:49.905734+0000","last_active":"2024-04-26T20:21:50.888715+0000","last_peered":"2024-04-26T20:21:50.888715+0000","last_clean":"2024-04-26T20:21:50.888715+0000","last_became_active":"2024-04-26T20:21:49.905424+0000","last_became_peered":"2024-04-26T20:21:49.905424+0000","last_unstale":"2024-04-26T20:21:50.888715+0000","last_undegraded":"2024-04-26T20:21:50.888715+0000","last_fullsized":"2024-04-26T20:21:50.888715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891379+0000","last_change":"2024-04-26T20:21:48.877043+0000","last_active":"2024-04-26T20:21:50.891379+0000","last_peered":"2024-04-26T20:21:50.891379+0000","last_clean":"2024-04-26T20:21:50.891379+0000","last_became_active":"2024-04-26T20:21:48.876721+0000","last_became_peered":"2024-04-26T20:21:48.876721+0000","last_unstale":"2024-04-26T20:21:50.891379+0000","last_undegraded":"2024-04-26T20:21:50.891379+0000","last_fullsized":"2024-04-26T20:21:50.891379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317752+0000","last_change":"2024-04-26T20:21:49.890323+0000","last_active":"2024-04-26T20:21:51.317752+0000","last_peered":"2024-04-26T20:21:51.317752+0000","last_clean":"2024-04-26T20:21:51.317752+0000","last_became_active":"2024-04-26T20:21:49.889790+0000","last_became_peered":"2024-04-26T20:21:49.889790+0000","last_unstale":"2024-04-26T20:21:51.317752+0000","last_undegraded":"2024-04-26T20:21:51.317752+0000","last_fullsized":"2024-04-26T20:21:51.317752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:27.648550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317863+0000","last_change":"2024-04-26T20:21:48.872759+0000","last_active":"2024-04-26T20:21:51.317863+0000","last_peered":"2024-04-26T20:21:51.317863+0000","last_clean":"2024-04-26T20:21:51.317863+0000","last_became_active":"2024-04-26T20:21:48.872461+0000","last_became_peered":"2024-04-26T20:21:48.872461+0000","last_unstale":"2024-04-26T20:21:51.317863+0000","last_undegraded":"2024-04-26T20:21:51.317863+0000","last_fullsized":"2024-04-26T20:21:51.317863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888264+0000","last_change":"2024-04-26T20:21:49.885391+0000","last_active":"2024-04-26T20:21:50.888264+0000","last_peered":"2024-04-26T20:21:50.888264+0000","last_clean":"2024-04-26T20:21:50.888264+0000","last_became_active":"2024-04-26T20:21:49.884866+0000","last_became_peered":"2024-04-26T20:21:49.884866+0000","last_unstale":"2024-04-26T20:21:50.888264+0000","last_undegraded":"2024-04-26T20:21:50.888264+0000","last_fullsized":"2024-04-26T20:21:50.888264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888983+0000","last_change":"2024-04-26T20:21:48.873334+0000","last_active":"2024-04-26T20:21:50.888983+0000","last_peered":"2024-04-26T20:21:50.888983+0000","last_clean":"2024-04-26T20:21:50.888983+0000","last_became_active":"2024-04-26T20:21:48.872910+0000","last_became_peered":"2024-04-26T20:21:48.872910+0000","last_unstale":"2024-04-26T20:21:50.888983+0000","last_undegraded":"2024-04-26T20:21:50.888983+0000","last_fullsized":"2024-04-26T20:21:50.888983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:39.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317180+0000","last_change":"2024-04-26T20:21:49.889288+0000","last_active":"2024-04-26T20:21:51.317180+0000","last_peered":"2024-04-26T20:21:51.317180+0000","last_clean":"2024-04-26T20:21:51.317180+0000","last_became_active":"2024-04-26T20:21:49.889007+0000","last_became_peered":"2024-04-26T20:21:49.889007+0000","last_unstale":"2024-04-26T20:21:51.317180+0000","last_undegraded":"2024-04-26T20:21:51.317180+0000","last_fullsized":"2024-04-26T20:21:51.317180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:02.548978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891213+0000","last_change":"2024-04-26T20:21:48.875893+0000","last_active":"2024-04-26T20:21:50.891213+0000","last_peered":"2024-04-26T20:21:50.891213+0000","last_clean":"2024-04-26T20:21:50.891213+0000","last_became_active":"2024-04-26T20:21:48.875620+0000","last_became_peered":"2024-04-26T20:21:48.875620+0000","last_unstale":"2024-04-26T20:21:50.891213+0000","last_undegraded":"2024-04-26T20:21:50.891213+0000","last_fullsized":"2024-04-26T20:21:50.891213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317151+0000","last_change":"2024-04-26T20:21:49.895277+0000","last_active":"2024-04-26T20:21:51.317151+0000","last_peered":"2024-04-26T20:21:51.317151+0000","last_clean":"2024-04-26T20:21:51.317151+0000","last_became_active":"2024-04-26T20:21:49.894988+0000","last_became_peered":"2024-04-26T20:21:49.894988+0000","last_unstale":"2024-04-26T20:21:51.317151+0000","last_undegraded":"2024-04-26T20:21:51.317151+0000","last_fullsized":"2024-04-26T20:21:51.317151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:32.476892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888386+0000","last_change":"2024-04-26T20:21:48.881818+0000","last_active":"2024-04-26T20:21:50.888386+0000","last_peered":"2024-04-26T20:21:50.888386+0000","last_clean":"2024-04-26T20:21:50.888386+0000","last_became_active":"2024-04-26T20:21:48.881426+0000","last_became_peered":"2024-04-26T20:21:48.881426+0000","last_unstale":"2024-04-26T20:21:50.888386+0000","last_undegraded":"2024-04-26T20:21:50.888386+0000","last_fullsized":"2024-04-26T20:21:50.888386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:10.468097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888081+0000","last_change":"2024-04-26T20:21:49.891143+0000","last_active":"2024-04-26T20:21:50.888081+0000","last_peered":"2024-04-26T20:21:50.888081+0000","last_clean":"2024-04-26T20:21:50.888081+0000","last_became_active":"2024-04-26T20:21:49.890710+0000","last_became_peered":"2024-04-26T20:21:49.890710+0000","last_unstale":"2024-04-26T20:21:50.888081+0000","last_undegraded":"2024-04-26T20:21:50.888081+0000","last_fullsized":"2024-04-26T20:21:50.888081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318330+0000","last_change":"2024-04-26T20:21:48.872822+0000","last_active":"2024-04-26T20:21:51.318330+0000","last_peered":"2024-04-26T20:21:51.318330+0000","last_clean":"2024-04-26T20:21:51.318330+0000","last_became_active":"2024-04-26T20:21:48.872542+0000","last_became_peered":"2024-04-26T20:21:48.872542+0000","last_unstale":"2024-04-26T20:21:51.318330+0000","last_undegraded":"2024-04-26T20:21:51.318330+0000","last_fullsized":"2024-04-26T20:21:51.318330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318093+0000","last_change":"2024-04-26T20:21:49.894321+0000","last_active":"2024-04-26T20:21:51.318093+0000","last_peered":"2024-04-26T20:21:51.318093+0000","last_clean":"2024-04-26T20:21:51.318093+0000","last_became_active":"2024-04-26T20:21:49.893996+0000","last_became_peered":"2024-04-26T20:21:49.893996+0000","last_unstale":"2024-04-26T20:21:51.318093+0000","last_undegraded":"2024-04-26T20:21:51.318093+0000","last_fullsized":"2024-04-26T20:21:51.318093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:52.770156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891066+0000","last_change":"2024-04-26T20:21:48.879859+0000","last_active":"2024-04-26T20:21:50.891066+0000","last_peered":"2024-04-26T20:21:50.891066+0000","last_clean":"2024-04-26T20:21:50.891066+0000","last_became_active":"2024-04-26T20:21:48.879557+0000","last_became_peered":"2024-04-26T20:21:48.879557+0000","last_unstale":"2024-04-26T20:21:50.891066+0000","last_undegraded":"2024-04-26T20:21:50.891066+0000","last_fullsized":"2024-04-26T20:21:50.891066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891233+0000","last_change":"2024-04-26T20:21:49.902684+0000","last_active":"2024-04-26T20:21:50.891233+0000","last_peered":"2024-04-26T20:21:50.891233+0000","last_clean":"2024-04-26T20:21:50.891233+0000","last_became_active":"2024-04-26T20:21:49.902380+0000","last_became_peered":"2024-04-26T20:21:49.902380+0000","last_unstale":"2024-04-26T20:21:50.891233+0000","last_undegraded":"2024-04-26T20:21:50.891233+0000","last_fullsized":"2024-04-26T20:21:50.891233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890777+0000","last_change":"2024-04-26T20:21:48.877680+0000","last_active":"2024-04-26T20:21:50.890777+0000","last_peered":"2024-04-26T20:21:50.890777+0000","last_clean":"2024-04-26T20:21:50.890777+0000","last_became_active":"2024-04-26T20:21:48.877390+0000","last_became_peered":"2024-04-26T20:21:48.877390+0000","last_unstale":"2024-04-26T20:21:50.890777+0000","last_undegraded":"2024-04-26T20:21:50.890777+0000","last_fullsized":"2024-04-26T20:21:50.890777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316851+0000","last_change":"2024-04-26T20:21:49.896604+0000","last_active":"2024-04-26T20:21:51.316851+0000","last_peered":"2024-04-26T20:21:51.316851+0000","last_clean":"2024-04-26T20:21:51.316851+0000","last_became_active":"2024-04-26T20:21:49.894327+0000","last_became_peered":"2024-04-26T20:21:49.894327+0000","last_unstale":"2024-04-26T20:21:51.316851+0000","last_undegraded":"2024-04-26T20:21:51.316851+0000","last_fullsized":"2024-04-26T20:21:51.316851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:46.887612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318312+0000","last_change":"2024-04-26T20:21:48.880981+0000","last_active":"2024-04-26T20:21:51.318312+0000","last_peered":"2024-04-26T20:21:51.318312+0000","last_clean":"2024-04-26T20:21:51.318312+0000","last_became_active":"2024-04-26T20:21:48.880662+0000","last_became_peered":"2024-04-26T20:21:48.880662+0000","last_unstale":"2024-04-26T20:21:51.318312+0000","last_undegraded":"2024-04-26T20:21:51.318312+0000","last_fullsized":"2024-04-26T20:21:51.318312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:12.587571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317595+0000","last_change":"2024-04-26T20:21:49.894131+0000","last_active":"2024-04-26T20:21:51.317595+0000","last_peered":"2024-04-26T20:21:51.317595+0000","last_clean":"2024-04-26T20:21:51.317595+0000","last_became_active":"2024-04-26T20:21:49.893827+0000","last_became_peered":"2024-04-26T20:21:49.893827+0000","last_unstale":"2024-04-26T20:21:51.317595+0000","last_undegraded":"2024-04-26T20:21:51.317595+0000","last_fullsized":"2024-04-26T20:21:51.317595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:57.212796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890988+0000","last_change":"2024-04-26T20:21:48.875201+0000","last_active":"2024-04-26T20:21:50.890988+0000","last_peered":"2024-04-26T20:21:50.890988+0000","last_clean":"2024-04-26T20:21:50.890988+0000","last_became_active":"2024-04-26T20:21:48.874859+0000","last_became_peered":"2024-04-26T20:21:48.874859+0000","last_unstale":"2024-04-26T20:21:50.890988+0000","last_undegraded":"2024-04-26T20:21:50.890988+0000","last_fullsized":"2024-04-26T20:21:50.890988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317768+0000","last_change":"2024-04-26T20:21:49.893283+0000","last_active":"2024-04-26T20:21:51.317768+0000","last_peered":"2024-04-26T20:21:51.317768+0000","last_clean":"2024-04-26T20:21:51.317768+0000","last_became_active":"2024-04-26T20:21:49.893006+0000","last_became_peered":"2024-04-26T20:21:49.893006+0000","last_unstale":"2024-04-26T20:21:51.317768+0000","last_undegraded":"2024-04-26T20:21:51.317768+0000","last_fullsized":"2024-04-26T20:21:51.317768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:57.391564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888957+0000","last_change":"2024-04-26T20:21:48.880076+0000","last_active":"2024-04-26T20:21:50.888957+0000","last_peered":"2024-04-26T20:21:50.888957+0000","last_clean":"2024-04-26T20:21:50.888957+0000","last_became_active":"2024-04-26T20:21:48.879805+0000","last_became_peered":"2024-04-26T20:21:48.879805+0000","last_unstale":"2024-04-26T20:21:50.888957+0000","last_undegraded":"2024-04-26T20:21:50.888957+0000","last_fullsized":"2024-04-26T20:21:50.888957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:25.685618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318491+0000","last_change":"2024-04-26T20:21:49.895219+0000","last_active":"2024-04-26T20:21:51.318491+0000","last_peered":"2024-04-26T20:21:51.318491+0000","last_clean":"2024-04-26T20:21:51.318491+0000","last_became_active":"2024-04-26T20:21:49.894965+0000","last_became_peered":"2024-04-26T20:21:49.894965+0000","last_unstale":"2024-04-26T20:21:51.318491+0000","last_undegraded":"2024-04-26T20:21:51.318491+0000","last_fullsized":"2024-04-26T20:21:51.318491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:50.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890958+0000","last_change":"2024-04-26T20:21:48.878737+0000","last_active":"2024-04-26T20:21:50.890958+0000","last_peered":"2024-04-26T20:21:50.890958+0000","last_clean":"2024-04-26T20:21:50.890958+0000","last_became_active":"2024-04-26T20:21:48.878184+0000","last_became_peered":"2024-04-26T20:21:48.878184+0000","last_unstale":"2024-04-26T20:21:50.890958+0000","last_undegraded":"2024-04-26T20:21:50.890958+0000","last_fullsized":"2024-04-26T20:21:50.890958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:40:48.030416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317314+0000","last_change":"2024-04-26T20:21:49.897731+0000","last_active":"2024-04-26T20:21:51.317314+0000","last_peered":"2024-04-26T20:21:51.317314+0000","last_clean":"2024-04-26T20:21:51.317314+0000","last_became_active":"2024-04-26T20:21:49.897398+0000","last_became_peered":"2024-04-26T20:21:49.897398+0000","last_unstale":"2024-04-26T20:21:51.317314+0000","last_undegraded":"2024-04-26T20:21:51.317314+0000","last_fullsized":"2024-04-26T20:21:51.317314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.737636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317884+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:21:51.317884+0000","last_peered":"2024-04-26T20:21:51.317884+0000","last_clean":"2024-04-26T20:21:51.317884+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:21:51.317884+0000","last_undegraded":"2024-04-26T20:21:51.317884+0000","last_fullsized":"2024-04-26T20:21:51.317884+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317283+0000","last_change":"2024-04-26T20:21:48.873546+0000","last_active":"2024-04-26T20:21:51.317283+0000","last_peered":"2024-04-26T20:21:51.317283+0000","last_clean":"2024-04-26T20:21:51.317283+0000","last_became_active":"2024-04-26T20:21:48.873258+0000","last_became_peered":"2024-04-26T20:21:48.873258+0000","last_unstale":"2024-04-26T20:21:51.317283+0000","last_undegraded":"2024-04-26T20:21:51.317283+0000","last_fullsized":"2024-04-26T20:21:51.317283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316468+0000","last_change":"2024-04-26T20:21:49.892620+0000","last_active":"2024-04-26T20:21:51.316468+0000","last_peered":"2024-04-26T20:21:51.316468+0000","last_clean":"2024-04-26T20:21:51.316468+0000","last_became_active":"2024-04-26T20:21:49.891979+0000","last_became_peered":"2024-04-26T20:21:49.891979+0000","last_unstale":"2024-04-26T20:21:51.316468+0000","last_undegraded":"2024-04-26T20:21:51.316468+0000","last_fullsized":"2024-04-26T20:21:51.316468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317818+0000","last_change":"2024-04-26T20:21:48.883411+0000","last_active":"2024-04-26T20:21:51.317818+0000","last_peered":"2024-04-26T20:21:51.317818+0000","last_clean":"2024-04-26T20:21:51.317818+0000","last_became_active":"2024-04-26T20:21:48.883156+0000","last_became_peered":"2024-04-26T20:21:48.883156+0000","last_unstale":"2024-04-26T20:21:51.317818+0000","last_undegraded":"2024-04-26T20:21:51.317818+0000","last_fullsized":"2024-04-26T20:21:51.317818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:39.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888021+0000","last_change":"2024-04-26T20:21:49.889027+0000","last_active":"2024-04-26T20:21:50.888021+0000","last_peered":"2024-04-26T20:21:50.888021+0000","last_clean":"2024-04-26T20:21:50.888021+0000","last_became_active":"2024-04-26T20:21:49.888730+0000","last_became_peered":"2024-04-26T20:21:49.888730+0000","last_unstale":"2024-04-26T20:21:50.888021+0000","last_undegraded":"2024-04-26T20:21:50.888021+0000","last_fullsized":"2024-04-26T20:21:50.888021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890869+0000","last_change":"2024-04-26T20:21:48.875168+0000","last_active":"2024-04-26T20:21:50.890869+0000","last_peered":"2024-04-26T20:21:50.890869+0000","last_clean":"2024-04-26T20:21:50.890869+0000","last_became_active":"2024-04-26T20:21:48.874839+0000","last_became_peered":"2024-04-26T20:21:48.874839+0000","last_unstale":"2024-04-26T20:21:50.890869+0000","last_undegraded":"2024-04-26T20:21:50.890869+0000","last_fullsized":"2024-04-26T20:21:50.890869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888697+0000","last_change":"2024-04-26T20:21:49.895165+0000","last_active":"2024-04-26T20:21:50.888697+0000","last_peered":"2024-04-26T20:21:50.888697+0000","last_clean":"2024-04-26T20:21:50.888697+0000","last_became_active":"2024-04-26T20:21:49.894872+0000","last_became_peered":"2024-04-26T20:21:49.894872+0000","last_unstale":"2024-04-26T20:21:50.888697+0000","last_undegraded":"2024-04-26T20:21:50.888697+0000","last_fullsized":"2024-04-26T20:21:50.888697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318427+0000","last_change":"2024-04-26T20:21:48.874437+0000","last_active":"2024-04-26T20:21:51.318427+0000","last_peered":"2024-04-26T20:21:51.318427+0000","last_clean":"2024-04-26T20:21:51.318427+0000","last_became_active":"2024-04-26T20:21:48.874165+0000","last_became_peered":"2024-04-26T20:21:48.874165+0000","last_unstale":"2024-04-26T20:21:51.318427+0000","last_undegraded":"2024-04-26T20:21:51.318427+0000","last_fullsized":"2024-04-26T20:21:51.318427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317263+0000","last_change":"2024-04-26T20:21:49.894448+0000","last_active":"2024-04-26T20:21:51.317263+0000","last_peered":"2024-04-26T20:21:51.317263+0000","last_clean":"2024-04-26T20:21:51.317263+0000","last_became_active":"2024-04-26T20:21:49.894119+0000","last_became_peered":"2024-04-26T20:21:49.894119+0000","last_unstale":"2024-04-26T20:21:51.317263+0000","last_undegraded":"2024-04-26T20:21:51.317263+0000","last_fullsized":"2024-04-26T20:21:51.317263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:40.680121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888896+0000","last_change":"2024-04-26T20:21:48.874713+0000","last_active":"2024-04-26T20:21:50.888896+0000","last_peered":"2024-04-26T20:21:50.888896+0000","last_clean":"2024-04-26T20:21:50.888896+0000","last_became_active":"2024-04-26T20:21:48.874414+0000","last_became_peered":"2024-04-26T20:21:48.874414+0000","last_unstale":"2024-04-26T20:21:50.888896+0000","last_undegraded":"2024-04-26T20:21:50.888896+0000","last_fullsized":"2024-04-26T20:21:50.888896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317980+0000","last_change":"2024-04-26T20:21:49.895171+0000","last_active":"2024-04-26T20:21:51.317980+0000","last_peered":"2024-04-26T20:21:51.317980+0000","last_clean":"2024-04-26T20:21:51.317980+0000","last_became_active":"2024-04-26T20:21:49.894889+0000","last_became_peered":"2024-04-26T20:21:49.894889+0000","last_unstale":"2024-04-26T20:21:51.317980+0000","last_undegraded":"2024-04-26T20:21:51.317980+0000","last_fullsized":"2024-04-26T20:21:51.317980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:46.887612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317633+0000","last_change":"2024-04-26T20:21:48.882622+0000","last_active":"2024-04-26T20:21:51.317633+0000","last_peered":"2024-04-26T20:21:51.317633+0000","last_clean":"2024-04-26T20:21:51.317633+0000","last_became_active":"2024-04-26T20:21:48.882344+0000","last_became_peered":"2024-04-26T20:21:48.882344+0000","last_unstale":"2024-04-26T20:21:51.317633+0000","last_undegraded":"2024-04-26T20:21:51.317633+0000","last_fullsized":"2024-04-26T20:21:51.317633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:35.034364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888822+0000","last_change":"2024-04-26T20:21:49.894444+0000","last_active":"2024-04-26T20:21:50.888822+0000","last_peered":"2024-04-26T20:21:50.888822+0000","last_clean":"2024-04-26T20:21:50.888822+0000","last_became_active":"2024-04-26T20:21:49.894111+0000","last_became_peered":"2024-04-26T20:21:49.894111+0000","last_unstale":"2024-04-26T20:21:50.888822+0000","last_undegraded":"2024-04-26T20:21:50.888822+0000","last_fullsized":"2024-04-26T20:21:50.888822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890978+0000","last_change":"2024-04-26T20:21:48.872831+0000","last_active":"2024-04-26T20:21:50.890978+0000","last_peered":"2024-04-26T20:21:50.890978+0000","last_clean":"2024-04-26T20:21:50.890978+0000","last_became_active":"2024-04-26T20:21:48.872523+0000","last_became_peered":"2024-04-26T20:21:48.872523+0000","last_unstale":"2024-04-26T20:21:50.890978+0000","last_undegraded":"2024-04-26T20:21:50.890978+0000","last_fullsized":"2024-04-26T20:21:50.890978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316932+0000","last_change":"2024-04-26T20:21:49.888957+0000","last_active":"2024-04-26T20:21:51.316932+0000","last_peered":"2024-04-26T20:21:51.316932+0000","last_clean":"2024-04-26T20:21:51.316932+0000","last_became_active":"2024-04-26T20:21:49.888621+0000","last_became_peered":"2024-04-26T20:21:49.888621+0000","last_unstale":"2024-04-26T20:21:51.316932+0000","last_undegraded":"2024-04-26T20:21:51.316932+0000","last_fullsized":"2024-04-26T20:21:51.316932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:53.734129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891334+0000","last_change":"2024-04-26T20:21:48.878830+0000","last_active":"2024-04-26T20:21:50.891334+0000","last_peered":"2024-04-26T20:21:50.891334+0000","last_clean":"2024-04-26T20:21:50.891334+0000","last_became_active":"2024-04-26T20:21:48.878560+0000","last_became_peered":"2024-04-26T20:21:48.878560+0000","last_unstale":"2024-04-26T20:21:50.891334+0000","last_undegraded":"2024-04-26T20:21:50.891334+0000","last_fullsized":"2024-04-26T20:21:50.891334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888017+0000","last_change":"2024-04-26T20:21:49.893919+0000","last_active":"2024-04-26T20:21:50.888017+0000","last_peered":"2024-04-26T20:21:50.888017+0000","last_clean":"2024-04-26T20:21:50.888017+0000","last_became_active":"2024-04-26T20:21:49.893508+0000","last_became_peered":"2024-04-26T20:21:49.893508+0000","last_unstale":"2024-04-26T20:21:50.888017+0000","last_undegraded":"2024-04-26T20:21:50.888017+0000","last_fullsized":"2024-04-26T20:21:50.888017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:42.131574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317700+0000","last_change":"2024-04-26T20:21:48.883061+0000","last_active":"2024-04-26T20:21:51.317700+0000","last_peered":"2024-04-26T20:21:51.317700+0000","last_clean":"2024-04-26T20:21:51.317700+0000","last_became_active":"2024-04-26T20:21:48.882776+0000","last_became_peered":"2024-04-26T20:21:48.882776+0000","last_unstale":"2024-04-26T20:21:51.317700+0000","last_undegraded":"2024-04-26T20:21:51.317700+0000","last_fullsized":"2024-04-26T20:21:51.317700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316756+0000","last_change":"2024-04-26T20:21:49.892633+0000","last_active":"2024-04-26T20:21:51.316756+0000","last_peered":"2024-04-26T20:21:51.316756+0000","last_clean":"2024-04-26T20:21:51.316756+0000","last_became_active":"2024-04-26T20:21:49.892308+0000","last_became_peered":"2024-04-26T20:21:49.892308+0000","last_unstale":"2024-04-26T20:21:51.316756+0000","last_undegraded":"2024-04-26T20:21:51.316756+0000","last_fullsized":"2024-04-26T20:21:51.316756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:59.560594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888824+0000","last_change":"2024-04-26T20:21:48.878718+0000","last_active":"2024-04-26T20:21:50.888824+0000","last_peered":"2024-04-26T20:21:50.888824+0000","last_clean":"2024-04-26T20:21:50.888824+0000","last_became_active":"2024-04-26T20:21:48.878382+0000","last_became_peered":"2024-04-26T20:21:48.878382+0000","last_unstale":"2024-04-26T20:21:50.888824+0000","last_undegraded":"2024-04-26T20:21:50.888824+0000","last_fullsized":"2024-04-26T20:21:50.888824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316423+0000","last_change":"2024-04-26T20:21:49.893414+0000","last_active":"2024-04-26T20:21:51.316423+0000","last_peered":"2024-04-26T20:21:51.316423+0000","last_clean":"2024-04-26T20:21:51.316423+0000","last_became_active":"2024-04-26T20:21:49.893094+0000","last_became_peered":"2024-04-26T20:21:49.893094+0000","last_unstale":"2024-04-26T20:21:51.316423+0000","last_undegraded":"2024-04-26T20:21:51.316423+0000","last_fullsized":"2024-04-26T20:21:51.316423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888774+0000","last_change":"2024-04-26T20:21:48.874766+0000","last_active":"2024-04-26T20:21:50.888774+0000","last_peered":"2024-04-26T20:21:50.888774+0000","last_clean":"2024-04-26T20:21:50.888774+0000","last_became_active":"2024-04-26T20:21:48.874339+0000","last_became_peered":"2024-04-26T20:21:48.874339+0000","last_unstale":"2024-04-26T20:21:50.888774+0000","last_undegraded":"2024-04-26T20:21:50.888774+0000","last_fullsized":"2024-04-26T20:21:50.888774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316594+0000","last_change":"2024-04-26T20:21:49.893713+0000","last_active":"2024-04-26T20:21:51.316594+0000","last_peered":"2024-04-26T20:21:51.316594+0000","last_clean":"2024-04-26T20:21:51.316594+0000","last_became_active":"2024-04-26T20:21:49.891376+0000","last_became_peered":"2024-04-26T20:21:49.891376+0000","last_unstale":"2024-04-26T20:21:51.316594+0000","last_undegraded":"2024-04-26T20:21:51.316594+0000","last_fullsized":"2024-04-26T20:21:51.316594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318283+0000","last_change":"2024-04-26T20:21:48.882680+0000","last_active":"2024-04-26T20:21:51.318283+0000","last_peered":"2024-04-26T20:21:51.318283+0000","last_clean":"2024-04-26T20:21:51.318283+0000","last_became_active":"2024-04-26T20:21:48.882393+0000","last_became_peered":"2024-04-26T20:21:48.882393+0000","last_unstale":"2024-04-26T20:21:51.318283+0000","last_undegraded":"2024-04-26T20:21:51.318283+0000","last_fullsized":"2024-04-26T20:21:51.318283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:41.422336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891134+0000","last_change":"2024-04-26T20:21:49.892761+0000","last_active":"2024-04-26T20:21:50.891134+0000","last_peered":"2024-04-26T20:21:50.891134+0000","last_clean":"2024-04-26T20:21:50.891134+0000","last_became_active":"2024-04-26T20:21:49.892403+0000","last_became_peered":"2024-04-26T20:21:49.892403+0000","last_unstale":"2024-04-26T20:21:50.891134+0000","last_undegraded":"2024-04-26T20:21:50.891134+0000","last_fullsized":"2024-04-26T20:21:50.891134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317815+0000","last_change":"2024-04-26T20:21:49.890488+0000","last_active":"2024-04-26T20:21:51.317815+0000","last_peered":"2024-04-26T20:21:51.317815+0000","last_clean":"2024-04-26T20:21:51.317815+0000","last_became_active":"2024-04-26T20:21:49.889950+0000","last_became_peered":"2024-04-26T20:21:49.889950+0000","last_unstale":"2024-04-26T20:21:51.317815+0000","last_undegraded":"2024-04-26T20:21:51.317815+0000","last_fullsized":"2024-04-26T20:21:51.317815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317232+0000","last_change":"2024-04-26T20:21:48.884202+0000","last_active":"2024-04-26T20:21:51.317232+0000","last_peered":"2024-04-26T20:21:51.317232+0000","last_clean":"2024-04-26T20:21:51.317232+0000","last_became_active":"2024-04-26T20:21:48.883945+0000","last_became_peered":"2024-04-26T20:21:48.883945+0000","last_unstale":"2024-04-26T20:21:51.317232+0000","last_undegraded":"2024-04-26T20:21:51.317232+0000","last_fullsized":"2024-04-26T20:21:51.317232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:25.685618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318377+0000","last_change":"2024-04-26T20:21:49.894650+0000","last_active":"2024-04-26T20:21:51.318377+0000","last_peered":"2024-04-26T20:21:51.318377+0000","last_clean":"2024-04-26T20:21:51.318377+0000","last_became_active":"2024-04-26T20:21:49.889860+0000","last_became_peered":"2024-04-26T20:21:49.889860+0000","last_unstale":"2024-04-26T20:21:51.318377+0000","last_undegraded":"2024-04-26T20:21:51.318377+0000","last_fullsized":"2024-04-26T20:21:51.318377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:13.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317454+0000","last_change":"2024-04-26T20:21:48.881647+0000","last_active":"2024-04-26T20:21:51.317454+0000","last_peered":"2024-04-26T20:21:51.317454+0000","last_clean":"2024-04-26T20:21:51.317454+0000","last_became_active":"2024-04-26T20:21:48.881349+0000","last_became_peered":"2024-04-26T20:21:48.881349+0000","last_unstale":"2024-04-26T20:21:51.317454+0000","last_undegraded":"2024-04-26T20:21:51.317454+0000","last_fullsized":"2024-04-26T20:21:51.317454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:24.800032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888111+0000","last_change":"2024-04-26T20:21:49.884044+0000","last_active":"2024-04-26T20:21:50.888111+0000","last_peered":"2024-04-26T20:21:50.888111+0000","last_clean":"2024-04-26T20:21:50.888111+0000","last_became_active":"2024-04-26T20:21:49.883750+0000","last_became_peered":"2024-04-26T20:21:49.883750+0000","last_unstale":"2024-04-26T20:21:50.888111+0000","last_undegraded":"2024-04-26T20:21:50.888111+0000","last_fullsized":"2024-04-26T20:21:50.888111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316858+0000","last_change":"2024-04-26T20:21:48.875662+0000","last_active":"2024-04-26T20:21:51.316858+0000","last_peered":"2024-04-26T20:21:51.316858+0000","last_clean":"2024-04-26T20:21:51.316858+0000","last_became_active":"2024-04-26T20:21:48.875024+0000","last_became_peered":"2024-04-26T20:21:48.875024+0000","last_unstale":"2024-04-26T20:21:51.316858+0000","last_undegraded":"2024-04-26T20:21:51.316858+0000","last_fullsized":"2024-04-26T20:21:51.316858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:40:48.030416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316941+0000","last_change":"2024-04-26T20:21:49.894476+0000","last_active":"2024-04-26T20:21:51.316941+0000","last_peered":"2024-04-26T20:21:51.316941+0000","last_clean":"2024-04-26T20:21:51.316941+0000","last_became_active":"2024-04-26T20:21:49.894186+0000","last_became_peered":"2024-04-26T20:21:49.894186+0000","last_unstale":"2024-04-26T20:21:51.316941+0000","last_undegraded":"2024-04-26T20:21:51.316941+0000","last_fullsized":"2024-04-26T20:21:51.316941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:27.648550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888239+0000","last_change":"2024-04-26T20:21:48.880562+0000","last_active":"2024-04-26T20:21:50.888239+0000","last_peered":"2024-04-26T20:21:50.888239+0000","last_clean":"2024-04-26T20:21:50.888239+0000","last_became_active":"2024-04-26T20:21:48.880249+0000","last_became_peered":"2024-04-26T20:21:48.880249+0000","last_unstale":"2024-04-26T20:21:50.888239+0000","last_undegraded":"2024-04-26T20:21:50.888239+0000","last_fullsized":"2024-04-26T20:21:50.888239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:21:59.574 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:21:59.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.709+0000 7fcabaa37640 1 Processor -- start 2024-04-26T20:21:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.709+0000 7fcabaa37640 1 -- start start 2024-04-26T20:21:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.710+0000 7fcabaa37640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcab415f380 0x7fcab40d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.710+0000 7fcabaa37640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcab415f760 0x7fcab40d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.710+0000 7fcabaa37640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcab40d5b50 0x7fcab40cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.710+0000 7fcabaa37640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fcab40d43a0 con 0x7fcab415f760 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.710+0000 7fcabaa37640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcab40d3de0 con 0x7fcab415f380 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.710+0000 7fcabaa37640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcab40d40c0 con 0x7fcab40d5b50 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.710+0000 7fcab37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcab415f760 0x7fcab40d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.710+0000 7fcab37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcab415f760 0x7fcab40d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35242/0 (socket says 172.21.15.37:35242) 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.710+0000 7fcab37fe640 1 -- 172.21.15.37:0/2588866139 learned_addr learned my addr 172.21.15.37:0/2588866139 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.710+0000 7fcab3fff640 1 --2- 172.21.15.37:0/2588866139 >> v2:172.21.15.134:3300/0 conn(0x7fcab415f380 0x7fcab40d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.710+0000 7fcab8fd4640 1 --2- 172.21.15.37:0/2588866139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcab40d5b50 0x7fcab40cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.711+0000 7fcab37fe640 1 -- 172.21.15.37:0/2588866139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcab40d5b50 msgr2=0x7fcab40cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.711+0000 7fcab37fe640 1 --2- 172.21.15.37:0/2588866139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcab40d5b50 0x7fcab40cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.711+0000 7fcab37fe640 1 -- 172.21.15.37:0/2588866139 >> v2:172.21.15.134:3300/0 conn(0x7fcab415f380 msgr2=0x7fcab40d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.711+0000 7fcab37fe640 1 --2- 172.21.15.37:0/2588866139 >> v2:172.21.15.134:3300/0 conn(0x7fcab415f380 0x7fcab40d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.711+0000 7fcab37fe640 1 -- 172.21.15.37:0/2588866139 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab40b8280 con 0x7fcab415f760 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.711+0000 7fcab37fe640 1 --2- 172.21.15.37:0/2588866139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab415f760 0x7fcab40d2110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcab408eb70 tx=0x7fcaa402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e905e6b960c95f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.711+0000 7fcab2ffd640 1 -- 172.21.15.37:0/2588866139 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4037070 con 0x7fcab415f760 2024-04-26T20:21:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.711+0000 7fcab2ffd640 1 -- 172.21.15.37:0/2588866139 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa4032440 con 0x7fcab415f760 2024-04-26T20:21:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.711+0000 7fcab2ffd640 1 -- 172.21.15.37:0/2588866139 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4032810 con 0x7fcab415f760 2024-04-26T20:21:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.712+0000 7fcabaa37640 1 -- 172.21.15.37:0/2588866139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab415f760 msgr2=0x7fcab40d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.712+0000 7fcabaa37640 1 --2- 172.21.15.37:0/2588866139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab415f760 0x7fcab40d2110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcab408eb70 tx=0x7fcaa402a270 comp rx=0 tx=0).stop 2024-04-26T20:21:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.712+0000 7fcabaa37640 1 -- 172.21.15.37:0/2588866139 shutdown_connections 2024-04-26T20:21:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.712+0000 7fcabaa37640 1 --2- 172.21.15.37:0/2588866139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcab40d5b50 0x7fcab40cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.712+0000 7fcabaa37640 1 --2- 172.21.15.37:0/2588866139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab415f760 0x7fcab40d2110 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.712+0000 7fcabaa37640 1 --2- 172.21.15.37:0/2588866139 >> v2:172.21.15.134:3300/0 conn(0x7fcab415f380 0x7fcab40d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.712+0000 7fcabaa37640 1 -- 172.21.15.37:0/2588866139 >> 172.21.15.37:0/2588866139 conn(0x7fcab409e560 msgr2=0x7fcab40d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.712+0000 7fcabaa37640 1 -- 172.21.15.37:0/2588866139 shutdown_connections 2024-04-26T20:21:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.712+0000 7fcabaa37640 1 -- 172.21.15.37:0/2588866139 wait complete. 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcabaa37640 1 Processor -- start 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcabaa37640 1 -- start start 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcabaa37640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab40d5b50 0x7fcab4161de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcabaa37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab415f380 0x7fcab40ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcabaa37640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcab415f760 0x7fcab4150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcabaa37640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcab40d1280 con 0x7fcab415f380 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcabaa37640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcab40d1100 con 0x7fcab40d5b50 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcabaa37640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcab40d3840 con 0x7fcab415f760 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcab37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab415f380 0x7fcab40ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcab37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab415f380 0x7fcab40ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35246/0 (socket says 172.21.15.37:35246) 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcab37fe640 1 -- 172.21.15.37:0/1122819249 learned_addr learned my addr 172.21.15.37:0/1122819249 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcab3fff640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab40d5b50 0x7fcab4161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcab8fd4640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcab415f760 0x7fcab4150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcab37fe640 1 -- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcab415f760 msgr2=0x7fcab4150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcab37fe640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcab415f760 0x7fcab4150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcab37fe640 1 -- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab40d5b50 msgr2=0x7fcab4161de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcab37fe640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab40d5b50 0x7fcab4161de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.714+0000 7fcab37fe640 1 -- 172.21.15.37:0/1122819249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab41597f0 con 0x7fcab415f380 2024-04-26T20:21:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.715+0000 7fcab37fe640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab415f380 0x7fcab40ceb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcaa402a2b0 tx=0x7fcaa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.715+0000 7fcab0ff9640 1 -- 172.21.15.37:0/1122819249 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4041020 con 0x7fcab415f380 2024-04-26T20:21:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.715+0000 7fcab0ff9640 1 -- 172.21.15.37:0/1122819249 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa403d620 con 0x7fcab415f380 2024-04-26T20:21:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.715+0000 7fcabaa37640 1 -- 172.21.15.37:0/1122819249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab41607b0 con 0x7fcab415f380 2024-04-26T20:21:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.715+0000 7fcab0ff9640 1 -- 172.21.15.37:0/1122819249 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4037040 con 0x7fcab415f380 2024-04-26T20:21:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.715+0000 7fcabaa37640 1 -- 172.21.15.37:0/1122819249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab4160d70 con 0x7fcab415f380 2024-04-26T20:21:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.716+0000 7fcabaa37640 1 -- 172.21.15.37:0/1122819249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab406a360 con 0x7fcab415f380 2024-04-26T20:21:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.716+0000 7fcab0ff9640 1 -- 172.21.15.37:0/1122819249 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcaa4044020 con 0x7fcab415f380 2024-04-26T20:21:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.717+0000 7fcab0ff9640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaa006c8f0 0x7fcaa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.717+0000 7fcab0ff9640 1 -- 172.21.15.37:0/1122819249 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcaa40ac240 con 0x7fcab415f380 2024-04-26T20:21:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.717+0000 7fcab3fff640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaa006c8f0 0x7fcaa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.717+0000 7fcab3fff640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaa006c8f0 0x7fcaa006edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009ad0 tx=0x7fcaa8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.722+0000 7fcab0ff9640 1 -- 172.21.15.37:0/1122819249 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcaa404b030 con 0x7fcab415f380 2024-04-26T20:21:59.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.901+0000 7fcabaa37640 1 -- 172.21.15.37:0/1122819249 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcab4161950 con 0x7fcaa006c8f0 2024-04-26T20:21:59.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.907+0000 7fcab0ff9640 1 -- 172.21.15.37:0/1122819249 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fcab4161950 con 0x7fcaa006c8f0 2024-04-26T20:21:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:21:59.912 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 -- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaa006c8f0 msgr2=0x7fcaa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaa006c8f0 0x7fcaa006edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009ad0 tx=0x7fcaa8008040 comp rx=0 tx=0).stop 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 -- 172.21.15.37:0/1122819249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab415f380 msgr2=0x7fcab40ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab415f380 0x7fcab40ceb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcaa402a2b0 tx=0x7fcaa40046c0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 -- 172.21.15.37:0/1122819249 shutdown_connections 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaa006c8f0 0x7fcaa006edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcab415f760 0x7fcab4150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab415f380 0x7fcab40ceb30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 --2- 172.21.15.37:0/1122819249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab40d5b50 0x7fcab4161de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 -- 172.21.15.37:0/1122819249 >> 172.21.15.37:0/1122819249 conn(0x7fcab409e560 msgr2=0x7fcab40ba2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 -- 172.21.15.37:0/1122819249 shutdown_connections 2024-04-26T20:21:59.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:21:59.910+0000 7fcabaa37640 1 -- 172.21.15.37:0/1122819249 wait complete. 2024-04-26T20:21:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-26T20:21:59.336643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.465143"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317460+0000","last_change":"2024-04-26T20:21:49.899185+0000","last_active":"2024-04-26T20:21:51.317460+0000","last_peered":"2024-04-26T20:21:51.317460+0000","last_clean":"2024-04-26T20:21:51.317460+0000","last_became_active":"2024-04-26T20:21:49.898818+0000","last_became_peered":"2024-04-26T20:21:49.898818+0000","last_unstale":"2024-04-26T20:21:51.317460+0000","last_undegraded":"2024-04-26T20:21:51.317460+0000","last_fullsized":"2024-04-26T20:21:51.317460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:13.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.889053+0000","last_change":"2024-04-26T20:21:49.899484+0000","last_active":"2024-04-26T20:21:50.889053+0000","last_peered":"2024-04-26T20:21:50.889053+0000","last_clean":"2024-04-26T20:21:50.889053+0000","last_became_active":"2024-04-26T20:21:49.899091+0000","last_became_peered":"2024-04-26T20:21:49.899091+0000","last_unstale":"2024-04-26T20:21:50.889053+0000","last_undegraded":"2024-04-26T20:21:50.889053+0000","last_fullsized":"2024-04-26T20:21:50.889053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.887979+0000","last_change":"2024-04-26T20:21:49.881814+0000","last_active":"2024-04-26T20:21:50.887979+0000","last_peered":"2024-04-26T20:21:50.887979+0000","last_clean":"2024-04-26T20:21:50.887979+0000","last_became_active":"2024-04-26T20:21:49.881507+0000","last_became_peered":"2024-04-26T20:21:49.881507+0000","last_unstale":"2024-04-26T20:21:50.887979+0000","last_undegraded":"2024-04-26T20:21:50.887979+0000","last_fullsized":"2024-04-26T20:21:50.887979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:30.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318197+0000","last_change":"2024-04-26T20:21:49.896399+0000","last_active":"2024-04-26T20:21:51.318197+0000","last_peered":"2024-04-26T20:21:51.318197+0000","last_clean":"2024-04-26T20:21:51.318197+0000","last_became_active":"2024-04-26T20:21:49.896136+0000","last_became_peered":"2024-04-26T20:21:49.896136+0000","last_unstale":"2024-04-26T20:21:51.318197+0000","last_undegraded":"2024-04-26T20:21:51.318197+0000","last_fullsized":"2024-04-26T20:21:51.318197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.737636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317925+0000","last_change":"2024-04-26T20:21:49.898821+0000","last_active":"2024-04-26T20:21:51.317925+0000","last_peered":"2024-04-26T20:21:51.317925+0000","last_clean":"2024-04-26T20:21:51.317925+0000","last_became_active":"2024-04-26T20:21:49.898554+0000","last_became_peered":"2024-04-26T20:21:49.898554+0000","last_unstale":"2024-04-26T20:21:51.317925+0000","last_undegraded":"2024-04-26T20:21:51.317925+0000","last_fullsized":"2024-04-26T20:21:51.317925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:52.618628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316822+0000","last_change":"2024-04-26T20:21:49.878509+0000","last_active":"2024-04-26T20:21:51.316822+0000","last_peered":"2024-04-26T20:21:51.316822+0000","last_clean":"2024-04-26T20:21:51.316822+0000","last_became_active":"2024-04-26T20:21:49.878237+0000","last_became_peered":"2024-04-26T20:21:49.878237+0000","last_unstale":"2024-04-26T20:21:51.316822+0000","last_undegraded":"2024-04-26T20:21:51.316822+0000","last_fullsized":"2024-04-26T20:21:51.316822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316684+0000","last_change":"2024-04-26T20:21:49.898211+0000","last_active":"2024-04-26T20:21:51.316684+0000","last_peered":"2024-04-26T20:21:51.316684+0000","last_clean":"2024-04-26T20:21:51.316684+0000","last_became_active":"2024-04-26T20:21:49.897952+0000","last_became_peered":"2024-04-26T20:21:49.897952+0000","last_unstale":"2024-04-26T20:21:51.316684+0000","last_undegraded":"2024-04-26T20:21:51.316684+0000","last_fullsized":"2024-04-26T20:21:51.316684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316222+0000","last_change":"2024-04-26T20:21:49.892897+0000","last_active":"2024-04-26T20:21:51.316222+0000","last_peered":"2024-04-26T20:21:51.316222+0000","last_clean":"2024-04-26T20:21:51.316222+0000","last_became_active":"2024-04-26T20:21:49.890640+0000","last_became_peered":"2024-04-26T20:21:49.890640+0000","last_unstale":"2024-04-26T20:21:51.316222+0000","last_undegraded":"2024-04-26T20:21:51.316222+0000","last_fullsized":"2024-04-26T20:21:51.316222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:42.131574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317597+0000","last_change":"2024-04-26T20:21:49.901710+0000","last_active":"2024-04-26T20:21:51.317597+0000","last_peered":"2024-04-26T20:21:51.317597+0000","last_clean":"2024-04-26T20:21:51.317597+0000","last_became_active":"2024-04-26T20:21:49.901363+0000","last_became_peered":"2024-04-26T20:21:49.901363+0000","last_unstale":"2024-04-26T20:21:51.317597+0000","last_undegraded":"2024-04-26T20:21:51.317597+0000","last_fullsized":"2024-04-26T20:21:51.317597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:50.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891437+0000","last_change":"2024-04-26T20:21:49.900636+0000","last_active":"2024-04-26T20:21:50.891437+0000","last_peered":"2024-04-26T20:21:50.891437+0000","last_clean":"2024-04-26T20:21:50.891437+0000","last_became_active":"2024-04-26T20:21:49.900073+0000","last_became_peered":"2024-04-26T20:21:49.900073+0000","last_unstale":"2024-04-26T20:21:50.891437+0000","last_undegraded":"2024-04-26T20:21:50.891437+0000","last_fullsized":"2024-04-26T20:21:50.891437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891162+0000","last_change":"2024-04-26T20:21:49.892413+0000","last_active":"2024-04-26T20:21:50.891162+0000","last_peered":"2024-04-26T20:21:50.891162+0000","last_clean":"2024-04-26T20:21:50.891162+0000","last_became_active":"2024-04-26T20:21:49.892106+0000","last_became_peered":"2024-04-26T20:21:49.892106+0000","last_unstale":"2024-04-26T20:21:50.891162+0000","last_undegraded":"2024-04-26T20:21:50.891162+0000","last_fullsized":"2024-04-26T20:21:50.891162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317031+0000","last_change":"2024-04-26T20:21:49.894073+0000","last_active":"2024-04-26T20:21:51.317031+0000","last_peered":"2024-04-26T20:21:51.317031+0000","last_clean":"2024-04-26T20:21:51.317031+0000","last_became_active":"2024-04-26T20:21:49.893726+0000","last_became_peered":"2024-04-26T20:21:49.893726+0000","last_unstale":"2024-04-26T20:21:51.317031+0000","last_undegraded":"2024-04-26T20:21:51.317031+0000","last_fullsized":"2024-04-26T20:21:51.317031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:00.300212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316742+0000","last_change":"2024-04-26T20:21:49.895923+0000","last_active":"2024-04-26T20:21:51.316742+0000","last_peered":"2024-04-26T20:21:51.316742+0000","last_clean":"2024-04-26T20:21:51.316742+0000","last_became_active":"2024-04-26T20:21:49.895261+0000","last_became_peered":"2024-04-26T20:21:49.895261+0000","last_unstale":"2024-04-26T20:21:51.316742+0000","last_undegraded":"2024-04-26T20:21:51.316742+0000","last_fullsized":"2024-04-26T20:21:51.316742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.887942+0000","last_change":"2024-04-26T20:21:49.904360+0000","last_active":"2024-04-26T20:21:50.887942+0000","last_peered":"2024-04-26T20:21:50.887942+0000","last_clean":"2024-04-26T20:21:50.887942+0000","last_became_active":"2024-04-26T20:21:49.899936+0000","last_became_peered":"2024-04-26T20:21:49.899936+0000","last_unstale":"2024-04-26T20:21:50.887942+0000","last_undegraded":"2024-04-26T20:21:50.887942+0000","last_fullsized":"2024-04-26T20:21:50.887942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:55.987722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888860+0000","last_change":"2024-04-26T20:21:49.893445+0000","last_active":"2024-04-26T20:21:50.888860+0000","last_peered":"2024-04-26T20:21:50.888860+0000","last_clean":"2024-04-26T20:21:50.888860+0000","last_became_active":"2024-04-26T20:21:49.892704+0000","last_became_peered":"2024-04-26T20:21:49.892704+0000","last_unstale":"2024-04-26T20:21:50.888860+0000","last_undegraded":"2024-04-26T20:21:50.888860+0000","last_fullsized":"2024-04-26T20:21:50.888860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316915+0000","last_change":"2024-04-26T20:21:49.883172+0000","last_active":"2024-04-26T20:21:51.316915+0000","last_peered":"2024-04-26T20:21:51.316915+0000","last_clean":"2024-04-26T20:21:51.316915+0000","last_became_active":"2024-04-26T20:21:49.882916+0000","last_became_peered":"2024-04-26T20:21:49.882916+0000","last_unstale":"2024-04-26T20:21:51.316915+0000","last_undegraded":"2024-04-26T20:21:51.316915+0000","last_fullsized":"2024-04-26T20:21:51.316915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317389+0000","last_change":"2024-04-26T20:21:49.889742+0000","last_active":"2024-04-26T20:21:51.317389+0000","last_peered":"2024-04-26T20:21:51.317389+0000","last_clean":"2024-04-26T20:21:51.317389+0000","last_became_active":"2024-04-26T20:21:49.889417+0000","last_became_peered":"2024-04-26T20:21:49.889417+0000","last_unstale":"2024-04-26T20:21:51.317389+0000","last_undegraded":"2024-04-26T20:21:51.317389+0000","last_fullsized":"2024-04-26T20:21:51.317389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:53.734129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890942+0000","last_change":"2024-04-26T20:21:49.894387+0000","last_active":"2024-04-26T20:21:50.890942+0000","last_peered":"2024-04-26T20:21:50.890942+0000","last_clean":"2024-04-26T20:21:50.890942+0000","last_became_active":"2024-04-26T20:21:49.893464+0000","last_became_peered":"2024-04-26T20:21:49.893464+0000","last_unstale":"2024-04-26T20:21:50.890942+0000","last_undegraded":"2024-04-26T20:21:50.890942+0000","last_fullsized":"2024-04-26T20:21:50.890942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:26.689749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317646+0000","last_change":"2024-04-26T20:21:49.892977+0000","last_active":"2024-04-26T20:21:51.317646+0000","last_peered":"2024-04-26T20:21:51.317646+0000","last_clean":"2024-04-26T20:21:51.317646+0000","last_became_active":"2024-04-26T20:21:49.892709+0000","last_became_peered":"2024-04-26T20:21:49.892709+0000","last_unstale":"2024-04-26T20:21:51.317646+0000","last_undegraded":"2024-04-26T20:21:51.317646+0000","last_fullsized":"2024-04-26T20:21:51.317646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:44.689592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891273+0000","last_change":"2024-04-26T20:21:49.892717+0000","last_active":"2024-04-26T20:21:50.891273+0000","last_peered":"2024-04-26T20:21:50.891273+0000","last_clean":"2024-04-26T20:21:50.891273+0000","last_became_active":"2024-04-26T20:21:49.892029+0000","last_became_peered":"2024-04-26T20:21:49.892029+0000","last_unstale":"2024-04-26T20:21:50.891273+0000","last_undegraded":"2024-04-26T20:21:50.891273+0000","last_fullsized":"2024-04-26T20:21:50.891273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888152+0000","last_change":"2024-04-26T20:21:49.891937+0000","last_active":"2024-04-26T20:21:50.888152+0000","last_peered":"2024-04-26T20:21:50.888152+0000","last_clean":"2024-04-26T20:21:50.888152+0000","last_became_active":"2024-04-26T20:21:49.891631+0000","last_became_peered":"2024-04-26T20:21:49.891631+0000","last_unstale":"2024-04-26T20:21:50.888152+0000","last_undegraded":"2024-04-26T20:21:50.888152+0000","last_fullsized":"2024-04-26T20:21:50.888152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318190+0000","last_change":"2024-04-26T20:21:49.890285+0000","last_active":"2024-04-26T20:21:51.318190+0000","last_peered":"2024-04-26T20:21:51.318190+0000","last_clean":"2024-04-26T20:21:51.318190+0000","last_became_active":"2024-04-26T20:21:49.888800+0000","last_became_peered":"2024-04-26T20:21:49.888800+0000","last_unstale":"2024-04-26T20:21:51.318190+0000","last_undegraded":"2024-04-26T20:21:51.318190+0000","last_fullsized":"2024-04-26T20:21:51.318190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:32.476892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316465+0000","last_change":"2024-04-26T20:21:49.890345+0000","last_active":"2024-04-26T20:21:51.316465+0000","last_peered":"2024-04-26T20:21:51.316465+0000","last_clean":"2024-04-26T20:21:51.316465+0000","last_became_active":"2024-04-26T20:21:49.889999+0000","last_became_peered":"2024-04-26T20:21:49.889999+0000","last_unstale":"2024-04-26T20:21:51.316465+0000","last_undegraded":"2024-04-26T20:21:51.316465+0000","last_fullsized":"2024-04-26T20:21:51.316465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317064+0000","last_change":"2024-04-26T20:21:49.897497+0000","last_active":"2024-04-26T20:21:51.317064+0000","last_peered":"2024-04-26T20:21:51.317064+0000","last_clean":"2024-04-26T20:21:51.317064+0000","last_became_active":"2024-04-26T20:21:49.896968+0000","last_became_peered":"2024-04-26T20:21:49.896968+0000","last_unstale":"2024-04-26T20:21:51.317064+0000","last_undegraded":"2024-04-26T20:21:51.317064+0000","last_fullsized":"2024-04-26T20:21:51.317064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316953+0000","last_change":"2024-04-26T20:21:49.890916+0000","last_active":"2024-04-26T20:21:51.316953+0000","last_peered":"2024-04-26T20:21:51.316953+0000","last_clean":"2024-04-26T20:21:51.316953+0000","last_became_active":"2024-04-26T20:21:49.890517+0000","last_became_peered":"2024-04-26T20:21:49.890517+0000","last_unstale":"2024-04-26T20:21:51.316953+0000","last_undegraded":"2024-04-26T20:21:51.316953+0000","last_fullsized":"2024-04-26T20:21:51.316953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891447+0000","last_change":"2024-04-26T20:21:49.903237+0000","last_active":"2024-04-26T20:21:50.891447+0000","last_peered":"2024-04-26T20:21:50.891447+0000","last_clean":"2024-04-26T20:21:50.891447+0000","last_became_active":"2024-04-26T20:21:49.902924+0000","last_became_peered":"2024-04-26T20:21:49.902924+0000","last_unstale":"2024-04-26T20:21:50.891447+0000","last_undegraded":"2024-04-26T20:21:50.891447+0000","last_fullsized":"2024-04-26T20:21:50.891447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888809+0000","last_change":"2024-04-26T20:21:49.905100+0000","last_active":"2024-04-26T20:21:50.888809+0000","last_peered":"2024-04-26T20:21:50.888809+0000","last_clean":"2024-04-26T20:21:50.888809+0000","last_became_active":"2024-04-26T20:21:49.904037+0000","last_became_peered":"2024-04-26T20:21:49.904037+0000","last_unstale":"2024-04-26T20:21:50.888809+0000","last_undegraded":"2024-04-26T20:21:50.888809+0000","last_fullsized":"2024-04-26T20:21:50.888809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316677+0000","last_change":"2024-04-26T20:21:49.891939+0000","last_active":"2024-04-26T20:21:51.316677+0000","last_peered":"2024-04-26T20:21:51.316677+0000","last_clean":"2024-04-26T20:21:51.316677+0000","last_became_active":"2024-04-26T20:21:49.891662+0000","last_became_peered":"2024-04-26T20:21:49.891662+0000","last_unstale":"2024-04-26T20:21:51.316677+0000","last_undegraded":"2024-04-26T20:21:51.316677+0000","last_fullsized":"2024-04-26T20:21:51.316677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888299+0000","last_change":"2024-04-26T20:21:49.895730+0000","last_active":"2024-04-26T20:21:50.888299+0000","last_peered":"2024-04-26T20:21:50.888299+0000","last_clean":"2024-04-26T20:21:50.888299+0000","last_became_active":"2024-04-26T20:21:49.895244+0000","last_became_peered":"2024-04-26T20:21:49.895244+0000","last_unstale":"2024-04-26T20:21:50.888299+0000","last_undegraded":"2024-04-26T20:21:50.888299+0000","last_fullsized":"2024-04-26T20:21:50.888299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:26.044422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318438+0000","last_change":"2024-04-26T20:21:49.896367+0000","last_active":"2024-04-26T20:21:51.318438+0000","last_peered":"2024-04-26T20:21:51.318438+0000","last_clean":"2024-04-26T20:21:51.318438+0000","last_became_active":"2024-04-26T20:21:49.895963+0000","last_became_peered":"2024-04-26T20:21:49.895963+0000","last_unstale":"2024-04-26T20:21:51.318438+0000","last_undegraded":"2024-04-26T20:21:51.318438+0000","last_fullsized":"2024-04-26T20:21:51.318438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:57.212796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890871+0000","last_change":"2024-04-26T20:21:49.880643+0000","last_active":"2024-04-26T20:21:50.890871+0000","last_peered":"2024-04-26T20:21:50.890871+0000","last_clean":"2024-04-26T20:21:50.890871+0000","last_became_active":"2024-04-26T20:21:49.880295+0000","last_became_peered":"2024-04-26T20:21:49.880295+0000","last_unstale":"2024-04-26T20:21:50.890871+0000","last_undegraded":"2024-04-26T20:21:50.890871+0000","last_fullsized":"2024-04-26T20:21:50.890871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:40.680121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891215+0000","last_change":"2024-04-26T20:21:49.906308+0000","last_active":"2024-04-26T20:21:50.891215+0000","last_peered":"2024-04-26T20:21:50.891215+0000","last_clean":"2024-04-26T20:21:50.891215+0000","last_became_active":"2024-04-26T20:21:49.906092+0000","last_became_peered":"2024-04-26T20:21:49.906092+0000","last_unstale":"2024-04-26T20:21:50.891215+0000","last_undegraded":"2024-04-26T20:21:50.891215+0000","last_fullsized":"2024-04-26T20:21:50.891215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318011+0000","last_change":"2024-04-26T20:21:49.901086+0000","last_active":"2024-04-26T20:21:51.318011+0000","last_peered":"2024-04-26T20:21:51.318011+0000","last_clean":"2024-04-26T20:21:51.318011+0000","last_became_active":"2024-04-26T20:21:49.900797+0000","last_became_peered":"2024-04-26T20:21:49.900797+0000","last_unstale":"2024-04-26T20:21:51.318011+0000","last_undegraded":"2024-04-26T20:21:51.318011+0000","last_fullsized":"2024-04-26T20:21:51.318011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:26.689749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316705+0000","last_change":"2024-04-26T20:21:48.880856+0000","last_active":"2024-04-26T20:21:51.316705+0000","last_peered":"2024-04-26T20:21:51.316705+0000","last_clean":"2024-04-26T20:21:51.316705+0000","last_became_active":"2024-04-26T20:21:48.880512+0000","last_became_peered":"2024-04-26T20:21:48.880512+0000","last_unstale":"2024-04-26T20:21:51.316705+0000","last_undegraded":"2024-04-26T20:21:51.316705+0000","last_fullsized":"2024-04-26T20:21:51.316705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888381+0000","last_change":"2024-04-26T20:21:49.889888+0000","last_active":"2024-04-26T20:21:50.888381+0000","last_peered":"2024-04-26T20:21:50.888381+0000","last_clean":"2024-04-26T20:21:50.888381+0000","last_became_active":"2024-04-26T20:21:49.889451+0000","last_became_peered":"2024-04-26T20:21:49.889451+0000","last_unstale":"2024-04-26T20:21:50.888381+0000","last_undegraded":"2024-04-26T20:21:50.888381+0000","last_fullsized":"2024-04-26T20:21:50.888381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891210+0000","last_change":"2024-04-26T20:21:48.880648+0000","last_active":"2024-04-26T20:21:50.891210+0000","last_peered":"2024-04-26T20:21:50.891210+0000","last_clean":"2024-04-26T20:21:50.891210+0000","last_became_active":"2024-04-26T20:21:48.880389+0000","last_became_peered":"2024-04-26T20:21:48.880389+0000","last_unstale":"2024-04-26T20:21:50.891210+0000","last_undegraded":"2024-04-26T20:21:50.891210+0000","last_fullsized":"2024-04-26T20:21:50.891210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316415+0000","last_change":"2024-04-26T20:21:48.873199+0000","last_active":"2024-04-26T20:21:51.316415+0000","last_peered":"2024-04-26T20:21:51.316415+0000","last_clean":"2024-04-26T20:21:51.316415+0000","last_became_active":"2024-04-26T20:21:48.872868+0000","last_became_peered":"2024-04-26T20:21:48.872868+0000","last_unstale":"2024-04-26T20:21:51.316415+0000","last_undegraded":"2024-04-26T20:21:51.316415+0000","last_fullsized":"2024-04-26T20:21:51.316415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316315+0000","last_change":"2024-04-26T20:21:49.897450+0000","last_active":"2024-04-26T20:21:51.316315+0000","last_peered":"2024-04-26T20:21:51.316315+0000","last_clean":"2024-04-26T20:21:51.316315+0000","last_became_active":"2024-04-26T20:21:49.897165+0000","last_became_peered":"2024-04-26T20:21:49.897165+0000","last_unstale":"2024-04-26T20:21:51.316315+0000","last_undegraded":"2024-04-26T20:21:51.316315+0000","last_fullsized":"2024-04-26T20:21:51.316315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317170+0000","last_change":"2024-04-26T20:21:48.881023+0000","last_active":"2024-04-26T20:21:51.317170+0000","last_peered":"2024-04-26T20:21:51.317170+0000","last_clean":"2024-04-26T20:21:51.317170+0000","last_became_active":"2024-04-26T20:21:48.880610+0000","last_became_peered":"2024-04-26T20:21:48.880610+0000","last_unstale":"2024-04-26T20:21:51.317170+0000","last_undegraded":"2024-04-26T20:21:51.317170+0000","last_fullsized":"2024-04-26T20:21:51.317170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888715+0000","last_change":"2024-04-26T20:21:49.905734+0000","last_active":"2024-04-26T20:21:50.888715+0000","last_peered":"2024-04-26T20:21:50.888715+0000","last_clean":"2024-04-26T20:21:50.888715+0000","last_became_active":"2024-04-26T20:21:49.905424+0000","last_became_peered":"2024-04-26T20:21:49.905424+0000","last_unstale":"2024-04-26T20:21:50.888715+0000","last_undegraded":"2024-04-26T20:21:50.888715+0000","last_fullsized":"2024-04-26T20:21:50.888715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891379+0000","last_change":"2024-04-26T20:21:48.877043+0000","last_active":"2024-04-26T20:21:50.891379+0000","last_peered":"2024-04-26T20:21:50.891379+0000","last_clean":"2024-04-26T20:21:50.891379+0000","last_became_active":"2024-04-26T20:21:48.876721+0000","last_became_peered":"2024-04-26T20:21:48.876721+0000","last_unstale":"2024-04-26T20:21:50.891379+0000","last_undegraded":"2024-04-26T20:21:50.891379+0000","last_fullsized":"2024-04-26T20:21:50.891379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317752+0000","last_change":"2024-04-26T20:21:49.890323+0000","last_active":"2024-04-26T20:21:51.317752+0000","last_peered":"2024-04-26T20:21:51.317752+0000","last_clean":"2024-04-26T20:21:51.317752+0000","last_became_active":"2024-04-26T20:21:49.889790+0000","last_became_peered":"2024-04-26T20:21:49.889790+0000","last_unstale":"2024-04-26T20:21:51.317752+0000","last_undegraded":"2024-04-26T20:21:51.317752+0000","last_fullsized":"2024-04-26T20:21:51.317752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:27.648550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317863+0000","last_change":"2024-04-26T20:21:48.872759+0000","last_active":"2024-04-26T20:21:51.317863+0000","last_peered":"2024-04-26T20:21:51.317863+0000","last_clean":"2024-04-26T20:21:51.317863+0000","last_became_active":"2024-04-26T20:21:48.872461+0000","last_became_peered":"2024-04-26T20:21:48.872461+0000","last_unstale":"2024-04-26T20:21:51.317863+0000","last_undegraded":"2024-04-26T20:21:51.317863+0000","last_fullsized":"2024-04-26T20:21:51.317863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888264+0000","last_change":"2024-04-26T20:21:49.885391+0000","last_active":"2024-04-26T20:21:50.888264+0000","last_peered":"2024-04-26T20:21:50.888264+0000","last_clean":"2024-04-26T20:21:50.888264+0000","last_became_active":"2024-04-26T20:21:49.884866+0000","last_became_peered":"2024-04-26T20:21:49.884866+0000","last_unstale":"2024-04-26T20:21:50.888264+0000","last_undegraded":"2024-04-26T20:21:50.888264+0000","last_fullsized":"2024-04-26T20:21:50.888264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888983+0000","last_change":"2024-04-26T20:21:48.873334+0000","last_active":"2024-04-26T20:21:50.888983+0000","last_peered":"2024-04-26T20:21:50.888983+0000","last_clean":"2024-04-26T20:21:50.888983+0000","last_became_active":"2024-04-26T20:21:48.872910+0000","last_became_peered":"2024-04-26T20:21:48.872910+0000","last_unstale":"2024-04-26T20:21:50.888983+0000","last_undegraded":"2024-04-26T20:21:50.888983+0000","last_fullsized":"2024-04-26T20:21:50.888983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:39.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317180+0000","last_change":"2024-04-26T20:21:49.889288+0000","last_active":"2024-04-26T20:21:51.317180+0000","last_peered":"2024-04-26T20:21:51.317180+0000","last_clean":"2024-04-26T20:21:51.317180+0000","last_became_active":"2024-04-26T20:21:49.889007+0000","last_became_peered":"2024-04-26T20:21:49.889007+0000","last_unstale":"2024-04-26T20:21:51.317180+0000","last_undegraded":"2024-04-26T20:21:51.317180+0000","last_fullsized":"2024-04-26T20:21:51.317180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:02.548978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891213+0000","last_change":"2024-04-26T20:21:48.875893+0000","last_active":"2024-04-26T20:21:50.891213+0000","last_peered":"2024-04-26T20:21:50.891213+0000","last_clean":"2024-04-26T20:21:50.891213+0000","last_became_active":"2024-04-26T20:21:48.875620+0000","last_became_peered":"2024-04-26T20:21:48.875620+0000","last_unstale":"2024-04-26T20:21:50.891213+0000","last_undegraded":"2024-04-26T20:21:50.891213+0000","last_fullsized":"2024-04-26T20:21:50.891213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317151+0000","last_change":"2024-04-26T20:21:49.895277+0000","last_active":"2024-04-26T20:21:51.317151+0000","last_peered":"2024-04-26T20:21:51.317151+0000","last_clean":"2024-04-26T20:21:51.317151+0000","last_became_active":"2024-04-26T20:21:49.894988+0000","last_became_peered":"2024-04-26T20:21:49.894988+0000","last_unstale":"2024-04-26T20:21:51.317151+0000","last_undegraded":"2024-04-26T20:21:51.317151+0000","last_fullsized":"2024-04-26T20:21:51.317151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:32.476892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888386+0000","last_change":"2024-04-26T20:21:48.881818+0000","last_active":"2024-04-26T20:21:50.888386+0000","last_peered":"2024-04-26T20:21:50.888386+0000","last_clean":"2024-04-26T20:21:50.888386+0000","last_became_active":"2024-04-26T20:21:48.881426+0000","last_became_peered":"2024-04-26T20:21:48.881426+0000","last_unstale":"2024-04-26T20:21:50.888386+0000","last_undegraded":"2024-04-26T20:21:50.888386+0000","last_fullsized":"2024-04-26T20:21:50.888386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:10.468097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888081+0000","last_change":"2024-04-26T20:21:49.891143+0000","last_active":"2024-04-26T20:21:50.888081+0000","last_peered":"2024-04-26T20:21:50.888081+0000","last_clean":"2024-04-26T20:21:50.888081+0000","last_became_active":"2024-04-26T20:21:49.890710+0000","last_became_peered":"2024-04-26T20:21:49.890710+0000","last_unstale":"2024-04-26T20:21:50.888081+0000","last_undegraded":"2024-04-26T20:21:50.888081+0000","last_fullsized":"2024-04-26T20:21:50.888081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318330+0000","last_change":"2024-04-26T20:21:48.872822+0000","last_active":"2024-04-26T20:21:51.318330+0000","last_peered":"2024-04-26T20:21:51.318330+0000","last_clean":"2024-04-26T20:21:51.318330+0000","last_became_active":"2024-04-26T20:21:48.872542+0000","last_became_peered":"2024-04-26T20:21:48.872542+0000","last_unstale":"2024-04-26T20:21:51.318330+0000","last_undegraded":"2024-04-26T20:21:51.318330+0000","last_fullsized":"2024-04-26T20:21:51.318330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318093+0000","last_change":"2024-04-26T20:21:49.894321+0000","last_active":"2024-04-26T20:21:51.318093+0000","last_peered":"2024-04-26T20:21:51.318093+0000","last_clean":"2024-04-26T20:21:51.318093+0000","last_became_active":"2024-04-26T20:21:49.893996+0000","last_became_peered":"2024-04-26T20:21:49.893996+0000","last_unstale":"2024-04-26T20:21:51.318093+0000","last_undegraded":"2024-04-26T20:21:51.318093+0000","last_fullsized":"2024-04-26T20:21:51.318093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:52.770156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891066+0000","last_change":"2024-04-26T20:21:48.879859+0000","last_active":"2024-04-26T20:21:50.891066+0000","last_peered":"2024-04-26T20:21:50.891066+0000","last_clean":"2024-04-26T20:21:50.891066+0000","last_became_active":"2024-04-26T20:21:48.879557+0000","last_became_peered":"2024-04-26T20:21:48.879557+0000","last_unstale":"2024-04-26T20:21:50.891066+0000","last_undegraded":"2024-04-26T20:21:50.891066+0000","last_fullsized":"2024-04-26T20:21:50.891066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891233+0000","last_change":"2024-04-26T20:21:49.902684+0000","last_active":"2024-04-26T20:21:50.891233+0000","last_peered":"2024-04-26T20:21:50.891233+0000","last_clean":"2024-04-26T20:21:50.891233+0000","last_became_active":"2024-04-26T20:21:49.902380+0000","last_became_peered":"2024-04-26T20:21:49.902380+0000","last_unstale":"2024-04-26T20:21:50.891233+0000","last_undegraded":"2024-04-26T20:21:50.891233+0000","last_fullsized":"2024-04-26T20:21:50.891233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890777+0000","last_change":"2024-04-26T20:21:48.877680+0000","last_active":"2024-04-26T20:21:50.890777+0000","last_peered":"2024-04-26T20:21:50.890777+0000","last_clean":"2024-04-26T20:21:50.890777+0000","last_became_active":"2024-04-26T20:21:48.877390+0000","last_became_peered":"2024-04-26T20:21:48.877390+0000","last_unstale":"2024-04-26T20:21:50.890777+0000","last_undegraded":"2024-04-26T20:21:50.890777+0000","last_fullsized":"2024-04-26T20:21:50.890777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316851+0000","last_change":"2024-04-26T20:21:49.896604+0000","last_active":"2024-04-26T20:21:51.316851+0000","last_peered":"2024-04-26T20:21:51.316851+0000","last_clean":"2024-04-26T20:21:51.316851+0000","last_became_active":"2024-04-26T20:21:49.894327+0000","last_became_peered":"2024-04-26T20:21:49.894327+0000","last_unstale":"2024-04-26T20:21:51.316851+0000","last_undegraded":"2024-04-26T20:21:51.316851+0000","last_fullsized":"2024-04-26T20:21:51.316851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:46.887612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318312+0000","last_change":"2024-04-26T20:21:48.880981+0000","last_active":"2024-04-26T20:21:51.318312+0000","last_peered":"2024-04-26T20:21:51.318312+0000","last_clean":"2024-04-26T20:21:51.318312+0000","last_became_active":"2024-04-26T20:21:48.880662+0000","last_became_peered":"2024-04-26T20:21:48.880662+0000","last_unstale":"2024-04-26T20:21:51.318312+0000","last_undegraded":"2024-04-26T20:21:51.318312+0000","last_fullsized":"2024-04-26T20:21:51.318312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:12.587571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317595+0000","last_change":"2024-04-26T20:21:49.894131+0000","last_active":"2024-04-26T20:21:51.317595+0000","last_peered":"2024-04-26T20:21:51.317595+0000","last_clean":"2024-04-26T20:21:51.317595+0000","last_became_active":"2024-04-26T20:21:49.893827+0000","last_became_peered":"2024-04-26T20:21:49.893827+0000","last_unstale":"2024-04-26T20:21:51.317595+0000","last_undegraded":"2024-04-26T20:21:51.317595+0000","last_fullsized":"2024-04-26T20:21:51.317595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:57.212796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890988+0000","last_change":"2024-04-26T20:21:48.875201+0000","last_active":"2024-04-26T20:21:50.890988+0000","last_peered":"2024-04-26T20:21:50.890988+0000","last_clean":"2024-04-26T20:21:50.890988+0000","last_became_active":"2024-04-26T20:21:48.874859+0000","last_became_peered":"2024-04-26T20:21:48.874859+0000","last_unstale":"2024-04-26T20:21:50.890988+0000","last_undegraded":"2024-04-26T20:21:50.890988+0000","last_fullsized":"2024-04-26T20:21:50.890988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317768+0000","last_change":"2024-04-26T20:21:49.893283+0000","last_active":"2024-04-26T20:21:51.317768+0000","last_peered":"2024-04-26T20:21:51.317768+0000","last_clean":"2024-04-26T20:21:51.317768+0000","last_became_active":"2024-04-26T20:21:49.893006+0000","last_became_peered":"2024-04-26T20:21:49.893006+0000","last_unstale":"2024-04-26T20:21:51.317768+0000","last_undegraded":"2024-04-26T20:21:51.317768+0000","last_fullsized":"2024-04-26T20:21:51.317768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:57.391564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888957+0000","last_change":"2024-04-26T20:21:48.880076+0000","last_active":"2024-04-26T20:21:50.888957+0000","last_peered":"2024-04-26T20:21:50.888957+0000","last_clean":"2024-04-26T20:21:50.888957+0000","last_became_active":"2024-04-26T20:21:48.879805+0000","last_became_peered":"2024-04-26T20:21:48.879805+0000","last_unstale":"2024-04-26T20:21:50.888957+0000","last_undegraded":"2024-04-26T20:21:50.888957+0000","last_fullsized":"2024-04-26T20:21:50.888957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:25.685618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318491+0000","last_change":"2024-04-26T20:21:49.895219+0000","last_active":"2024-04-26T20:21:51.318491+0000","last_peered":"2024-04-26T20:21:51.318491+0000","last_clean":"2024-04-26T20:21:51.318491+0000","last_became_active":"2024-04-26T20:21:49.894965+0000","last_became_peered":"2024-04-26T20:21:49.894965+0000","last_unstale":"2024-04-26T20:21:51.318491+0000","last_undegraded":"2024-04-26T20:21:51.318491+0000","last_fullsized":"2024-04-26T20:21:51.318491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:50.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890958+0000","last_change":"2024-04-26T20:21:48.878737+0000","last_active":"2024-04-26T20:21:50.890958+0000","last_peered":"2024-04-26T20:21:50.890958+0000","last_clean":"2024-04-26T20:21:50.890958+0000","last_became_active":"2024-04-26T20:21:48.878184+0000","last_became_peered":"2024-04-26T20:21:48.878184+0000","last_unstale":"2024-04-26T20:21:50.890958+0000","last_undegraded":"2024-04-26T20:21:50.890958+0000","last_fullsized":"2024-04-26T20:21:50.890958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:40:48.030416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317314+0000","last_change":"2024-04-26T20:21:49.897731+0000","last_active":"2024-04-26T20:21:51.317314+0000","last_peered":"2024-04-26T20:21:51.317314+0000","last_clean":"2024-04-26T20:21:51.317314+0000","last_became_active":"2024-04-26T20:21:49.897398+0000","last_became_peered":"2024-04-26T20:21:49.897398+0000","last_unstale":"2024-04-26T20:21:51.317314+0000","last_undegraded":"2024-04-26T20:21:51.317314+0000","last_fullsized":"2024-04-26T20:21:51.317314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.737636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317884+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:21:51.317884+0000","last_peered":"2024-04-26T20:21:51.317884+0000","last_clean":"2024-04-26T20:21:51.317884+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:21:51.317884+0000","last_undegraded":"2024-04-26T20:21:51.317884+0000","last_fullsized":"2024-04-26T20:21:51.317884+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317283+0000","last_change":"2024-04-26T20:21:48.873546+0000","last_active":"2024-04-26T20:21:51.317283+0000","last_peered":"2024-04-26T20:21:51.317283+0000","last_clean":"2024-04-26T20:21:51.317283+0000","last_became_active":"2024-04-26T20:21:48.873258+0000","last_became_peered":"2024-04-26T20:21:48.873258+0000","last_unstale":"2024-04-26T20:21:51.317283+0000","last_undegraded":"2024-04-26T20:21:51.317283+0000","last_fullsized":"2024-04-26T20:21:51.317283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316468+0000","last_change":"2024-04-26T20:21:49.892620+0000","last_active":"2024-04-26T20:21:51.316468+0000","last_peered":"2024-04-26T20:21:51.316468+0000","last_clean":"2024-04-26T20:21:51.316468+0000","last_became_active":"2024-04-26T20:21:49.891979+0000","last_became_peered":"2024-04-26T20:21:49.891979+0000","last_unstale":"2024-04-26T20:21:51.316468+0000","last_undegraded":"2024-04-26T20:21:51.316468+0000","last_fullsized":"2024-04-26T20:21:51.316468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317818+0000","last_change":"2024-04-26T20:21:48.883411+0000","last_active":"2024-04-26T20:21:51.317818+0000","last_peered":"2024-04-26T20:21:51.317818+0000","last_clean":"2024-04-26T20:21:51.317818+0000","last_became_active":"2024-04-26T20:21:48.883156+0000","last_became_peered":"2024-04-26T20:21:48.883156+0000","last_unstale":"2024-04-26T20:21:51.317818+0000","last_undegraded":"2024-04-26T20:21:51.317818+0000","last_fullsized":"2024-04-26T20:21:51.317818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:39.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888021+0000","last_change":"2024-04-26T20:21:49.889027+0000","last_active":"2024-04-26T20:21:50.888021+0000","last_peered":"2024-04-26T20:21:50.888021+0000","last_clean":"2024-04-26T20:21:50.888021+0000","last_became_active":"2024-04-26T20:21:49.888730+0000","last_became_peered":"2024-04-26T20:21:49.888730+0000","last_unstale":"2024-04-26T20:21:50.888021+0000","last_undegraded":"2024-04-26T20:21:50.888021+0000","last_fullsized":"2024-04-26T20:21:50.888021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890869+0000","last_change":"2024-04-26T20:21:48.875168+0000","last_active":"2024-04-26T20:21:50.890869+0000","last_peered":"2024-04-26T20:21:50.890869+0000","last_clean":"2024-04-26T20:21:50.890869+0000","last_became_active":"2024-04-26T20:21:48.874839+0000","last_became_peered":"2024-04-26T20:21:48.874839+0000","last_unstale":"2024-04-26T20:21:50.890869+0000","last_undegraded":"2024-04-26T20:21:50.890869+0000","last_fullsized":"2024-04-26T20:21:50.890869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:50.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888697+0000","last_change":"2024-04-26T20:21:49.895165+0000","last_active":"2024-04-26T20:21:50.888697+0000","last_peered":"2024-04-26T20:21:50.888697+0000","last_clean":"2024-04-26T20:21:50.888697+0000","last_became_active":"2024-04-26T20:21:49.894872+0000","last_became_peered":"2024-04-26T20:21:49.894872+0000","last_unstale":"2024-04-26T20:21:50.888697+0000","last_undegraded":"2024-04-26T20:21:50.888697+0000","last_fullsized":"2024-04-26T20:21:50.888697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318427+0000","last_change":"2024-04-26T20:21:48.874437+0000","last_active":"2024-04-26T20:21:51.318427+0000","last_peered":"2024-04-26T20:21:51.318427+0000","last_clean":"2024-04-26T20:21:51.318427+0000","last_became_active":"2024-04-26T20:21:48.874165+0000","last_became_peered":"2024-04-26T20:21:48.874165+0000","last_unstale":"2024-04-26T20:21:51.318427+0000","last_undegraded":"2024-04-26T20:21:51.318427+0000","last_fullsized":"2024-04-26T20:21:51.318427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317263+0000","last_change":"2024-04-26T20:21:49.894448+0000","last_active":"2024-04-26T20:21:51.317263+0000","last_peered":"2024-04-26T20:21:51.317263+0000","last_clean":"2024-04-26T20:21:51.317263+0000","last_became_active":"2024-04-26T20:21:49.894119+0000","last_became_peered":"2024-04-26T20:21:49.894119+0000","last_unstale":"2024-04-26T20:21:51.317263+0000","last_undegraded":"2024-04-26T20:21:51.317263+0000","last_fullsized":"2024-04-26T20:21:51.317263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:40.680121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888896+0000","last_change":"2024-04-26T20:21:48.874713+0000","last_active":"2024-04-26T20:21:50.888896+0000","last_peered":"2024-04-26T20:21:50.888896+0000","last_clean":"2024-04-26T20:21:50.888896+0000","last_became_active":"2024-04-26T20:21:48.874414+0000","last_became_peered":"2024-04-26T20:21:48.874414+0000","last_unstale":"2024-04-26T20:21:50.888896+0000","last_undegraded":"2024-04-26T20:21:50.888896+0000","last_fullsized":"2024-04-26T20:21:50.888896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:59.296081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317980+0000","last_change":"2024-04-26T20:21:49.895171+0000","last_active":"2024-04-26T20:21:51.317980+0000","last_peered":"2024-04-26T20:21:51.317980+0000","last_clean":"2024-04-26T20:21:51.317980+0000","last_became_active":"2024-04-26T20:21:49.894889+0000","last_became_peered":"2024-04-26T20:21:49.894889+0000","last_unstale":"2024-04-26T20:21:51.317980+0000","last_undegraded":"2024-04-26T20:21:51.317980+0000","last_fullsized":"2024-04-26T20:21:51.317980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:46.887612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317633+0000","last_change":"2024-04-26T20:21:48.882622+0000","last_active":"2024-04-26T20:21:51.317633+0000","last_peered":"2024-04-26T20:21:51.317633+0000","last_clean":"2024-04-26T20:21:51.317633+0000","last_became_active":"2024-04-26T20:21:48.882344+0000","last_became_peered":"2024-04-26T20:21:48.882344+0000","last_unstale":"2024-04-26T20:21:51.317633+0000","last_undegraded":"2024-04-26T20:21:51.317633+0000","last_fullsized":"2024-04-26T20:21:51.317633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:35.034364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888822+0000","last_change":"2024-04-26T20:21:49.894444+0000","last_active":"2024-04-26T20:21:50.888822+0000","last_peered":"2024-04-26T20:21:50.888822+0000","last_clean":"2024-04-26T20:21:50.888822+0000","last_became_active":"2024-04-26T20:21:49.894111+0000","last_became_peered":"2024-04-26T20:21:49.894111+0000","last_unstale":"2024-04-26T20:21:50.888822+0000","last_undegraded":"2024-04-26T20:21:50.888822+0000","last_fullsized":"2024-04-26T20:21:50.888822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:26.082331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.890978+0000","last_change":"2024-04-26T20:21:48.872831+0000","last_active":"2024-04-26T20:21:50.890978+0000","last_peered":"2024-04-26T20:21:50.890978+0000","last_clean":"2024-04-26T20:21:50.890978+0000","last_became_active":"2024-04-26T20:21:48.872523+0000","last_became_peered":"2024-04-26T20:21:48.872523+0000","last_unstale":"2024-04-26T20:21:50.890978+0000","last_undegraded":"2024-04-26T20:21:50.890978+0000","last_fullsized":"2024-04-26T20:21:50.890978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316932+0000","last_change":"2024-04-26T20:21:49.888957+0000","last_active":"2024-04-26T20:21:51.316932+0000","last_peered":"2024-04-26T20:21:51.316932+0000","last_clean":"2024-04-26T20:21:51.316932+0000","last_became_active":"2024-04-26T20:21:49.888621+0000","last_became_peered":"2024-04-26T20:21:49.888621+0000","last_unstale":"2024-04-26T20:21:51.316932+0000","last_undegraded":"2024-04-26T20:21:51.316932+0000","last_fullsized":"2024-04-26T20:21:51.316932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:53.734129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891334+0000","last_change":"2024-04-26T20:21:48.878830+0000","last_active":"2024-04-26T20:21:50.891334+0000","last_peered":"2024-04-26T20:21:50.891334+0000","last_clean":"2024-04-26T20:21:50.891334+0000","last_became_active":"2024-04-26T20:21:48.878560+0000","last_became_peered":"2024-04-26T20:21:48.878560+0000","last_unstale":"2024-04-26T20:21:50.891334+0000","last_undegraded":"2024-04-26T20:21:50.891334+0000","last_fullsized":"2024-04-26T20:21:50.891334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:58.556463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888017+0000","last_change":"2024-04-26T20:21:49.893919+0000","last_active":"2024-04-26T20:21:50.888017+0000","last_peered":"2024-04-26T20:21:50.888017+0000","last_clean":"2024-04-26T20:21:50.888017+0000","last_became_active":"2024-04-26T20:21:49.893508+0000","last_became_peered":"2024-04-26T20:21:49.893508+0000","last_unstale":"2024-04-26T20:21:50.888017+0000","last_undegraded":"2024-04-26T20:21:50.888017+0000","last_fullsized":"2024-04-26T20:21:50.888017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:42.131574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317700+0000","last_change":"2024-04-26T20:21:48.883061+0000","last_active":"2024-04-26T20:21:51.317700+0000","last_peered":"2024-04-26T20:21:51.317700+0000","last_clean":"2024-04-26T20:21:51.317700+0000","last_became_active":"2024-04-26T20:21:48.882776+0000","last_became_peered":"2024-04-26T20:21:48.882776+0000","last_unstale":"2024-04-26T20:21:51.317700+0000","last_undegraded":"2024-04-26T20:21:51.317700+0000","last_fullsized":"2024-04-26T20:21:51.317700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316756+0000","last_change":"2024-04-26T20:21:49.892633+0000","last_active":"2024-04-26T20:21:51.316756+0000","last_peered":"2024-04-26T20:21:51.316756+0000","last_clean":"2024-04-26T20:21:51.316756+0000","last_became_active":"2024-04-26T20:21:49.892308+0000","last_became_peered":"2024-04-26T20:21:49.892308+0000","last_unstale":"2024-04-26T20:21:51.316756+0000","last_undegraded":"2024-04-26T20:21:51.316756+0000","last_fullsized":"2024-04-26T20:21:51.316756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:59.560594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888824+0000","last_change":"2024-04-26T20:21:48.878718+0000","last_active":"2024-04-26T20:21:50.888824+0000","last_peered":"2024-04-26T20:21:50.888824+0000","last_clean":"2024-04-26T20:21:50.888824+0000","last_became_active":"2024-04-26T20:21:48.878382+0000","last_became_peered":"2024-04-26T20:21:48.878382+0000","last_unstale":"2024-04-26T20:21:50.888824+0000","last_undegraded":"2024-04-26T20:21:50.888824+0000","last_fullsized":"2024-04-26T20:21:50.888824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:55.932252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316423+0000","last_change":"2024-04-26T20:21:49.893414+0000","last_active":"2024-04-26T20:21:51.316423+0000","last_peered":"2024-04-26T20:21:51.316423+0000","last_clean":"2024-04-26T20:21:51.316423+0000","last_became_active":"2024-04-26T20:21:49.893094+0000","last_became_peered":"2024-04-26T20:21:49.893094+0000","last_unstale":"2024-04-26T20:21:51.316423+0000","last_undegraded":"2024-04-26T20:21:51.316423+0000","last_fullsized":"2024-04-26T20:21:51.316423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:03.564211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888774+0000","last_change":"2024-04-26T20:21:48.874766+0000","last_active":"2024-04-26T20:21:50.888774+0000","last_peered":"2024-04-26T20:21:50.888774+0000","last_clean":"2024-04-26T20:21:50.888774+0000","last_became_active":"2024-04-26T20:21:48.874339+0000","last_became_peered":"2024-04-26T20:21:48.874339+0000","last_unstale":"2024-04-26T20:21:50.888774+0000","last_undegraded":"2024-04-26T20:21:50.888774+0000","last_fullsized":"2024-04-26T20:21:50.888774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:58:01.544847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316594+0000","last_change":"2024-04-26T20:21:49.893713+0000","last_active":"2024-04-26T20:21:51.316594+0000","last_peered":"2024-04-26T20:21:51.316594+0000","last_clean":"2024-04-26T20:21:51.316594+0000","last_became_active":"2024-04-26T20:21:49.891376+0000","last_became_peered":"2024-04-26T20:21:49.891376+0000","last_unstale":"2024-04-26T20:21:51.316594+0000","last_undegraded":"2024-04-26T20:21:51.316594+0000","last_fullsized":"2024-04-26T20:21:51.316594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:02.264311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318283+0000","last_change":"2024-04-26T20:21:48.882680+0000","last_active":"2024-04-26T20:21:51.318283+0000","last_peered":"2024-04-26T20:21:51.318283+0000","last_clean":"2024-04-26T20:21:51.318283+0000","last_became_active":"2024-04-26T20:21:48.882393+0000","last_became_peered":"2024-04-26T20:21:48.882393+0000","last_unstale":"2024-04-26T20:21:51.318283+0000","last_undegraded":"2024-04-26T20:21:51.318283+0000","last_fullsized":"2024-04-26T20:21:51.318283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:41.422336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.891134+0000","last_change":"2024-04-26T20:21:49.892761+0000","last_active":"2024-04-26T20:21:50.891134+0000","last_peered":"2024-04-26T20:21:50.891134+0000","last_clean":"2024-04-26T20:21:50.891134+0000","last_became_active":"2024-04-26T20:21:49.892403+0000","last_became_peered":"2024-04-26T20:21:49.892403+0000","last_unstale":"2024-04-26T20:21:50.891134+0000","last_undegraded":"2024-04-26T20:21:50.891134+0000","last_fullsized":"2024-04-26T20:21:50.891134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.490094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317815+0000","last_change":"2024-04-26T20:21:49.890488+0000","last_active":"2024-04-26T20:21:51.317815+0000","last_peered":"2024-04-26T20:21:51.317815+0000","last_clean":"2024-04-26T20:21:51.317815+0000","last_became_active":"2024-04-26T20:21:49.889950+0000","last_became_peered":"2024-04-26T20:21:49.889950+0000","last_unstale":"2024-04-26T20:21:51.317815+0000","last_undegraded":"2024-04-26T20:21:51.317815+0000","last_fullsized":"2024-04-26T20:21:51.317815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:00.384159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317232+0000","last_change":"2024-04-26T20:21:48.884202+0000","last_active":"2024-04-26T20:21:51.317232+0000","last_peered":"2024-04-26T20:21:51.317232+0000","last_clean":"2024-04-26T20:21:51.317232+0000","last_became_active":"2024-04-26T20:21:48.883945+0000","last_became_peered":"2024-04-26T20:21:48.883945+0000","last_unstale":"2024-04-26T20:21:51.317232+0000","last_undegraded":"2024-04-26T20:21:51.317232+0000","last_fullsized":"2024-04-26T20:21:51.317232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:25.685618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.318377+0000","last_change":"2024-04-26T20:21:49.894650+0000","last_active":"2024-04-26T20:21:51.318377+0000","last_peered":"2024-04-26T20:21:51.318377+0000","last_clean":"2024-04-26T20:21:51.318377+0000","last_became_active":"2024-04-26T20:21:49.889860+0000","last_became_peered":"2024-04-26T20:21:49.889860+0000","last_unstale":"2024-04-26T20:21:51.318377+0000","last_undegraded":"2024-04-26T20:21:51.318377+0000","last_fullsized":"2024-04-26T20:21:51.318377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:13.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.317454+0000","last_change":"2024-04-26T20:21:48.881647+0000","last_active":"2024-04-26T20:21:51.317454+0000","last_peered":"2024-04-26T20:21:51.317454+0000","last_clean":"2024-04-26T20:21:51.317454+0000","last_became_active":"2024-04-26T20:21:48.881349+0000","last_became_peered":"2024-04-26T20:21:48.881349+0000","last_unstale":"2024-04-26T20:21:51.317454+0000","last_undegraded":"2024-04-26T20:21:51.317454+0000","last_fullsized":"2024-04-26T20:21:51.317454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:24.800032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888111+0000","last_change":"2024-04-26T20:21:49.884044+0000","last_active":"2024-04-26T20:21:50.888111+0000","last_peered":"2024-04-26T20:21:50.888111+0000","last_clean":"2024-04-26T20:21:50.888111+0000","last_became_active":"2024-04-26T20:21:49.883750+0000","last_became_peered":"2024-04-26T20:21:49.883750+0000","last_unstale":"2024-04-26T20:21:50.888111+0000","last_undegraded":"2024-04-26T20:21:50.888111+0000","last_fullsized":"2024-04-26T20:21:50.888111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:19.637055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316858+0000","last_change":"2024-04-26T20:21:48.875662+0000","last_active":"2024-04-26T20:21:51.316858+0000","last_peered":"2024-04-26T20:21:51.316858+0000","last_clean":"2024-04-26T20:21:51.316858+0000","last_became_active":"2024-04-26T20:21:48.875024+0000","last_became_peered":"2024-04-26T20:21:48.875024+0000","last_unstale":"2024-04-26T20:21:51.316858+0000","last_undegraded":"2024-04-26T20:21:51.316858+0000","last_fullsized":"2024-04-26T20:21:51.316858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:40:48.030416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:51.316941+0000","last_change":"2024-04-26T20:21:49.894476+0000","last_active":"2024-04-26T20:21:51.316941+0000","last_peered":"2024-04-26T20:21:51.316941+0000","last_clean":"2024-04-26T20:21:51.316941+0000","last_became_active":"2024-04-26T20:21:49.894186+0000","last_became_peered":"2024-04-26T20:21:49.894186+0000","last_unstale":"2024-04-26T20:21:51.316941+0000","last_undegraded":"2024-04-26T20:21:51.316941+0000","last_fullsized":"2024-04-26T20:21:51.316941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:48.855798+0000","last_clean_scrub_stamp":"2024-04-26T20:21:48.855798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:27.648550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T20:21:50.888239+0000","last_change":"2024-04-26T20:21:48.880562+0000","last_active":"2024-04-26T20:21:50.888239+0000","last_peered":"2024-04-26T20:21:50.888239+0000","last_clean":"2024-04-26T20:21:50.888239+0000","last_became_active":"2024-04-26T20:21:48.880249+0000","last_became_peered":"2024-04-26T20:21:48.880249+0000","last_unstale":"2024-04-26T20:21:50.888239+0000","last_undegraded":"2024-04-26T20:21:50.888239+0000","last_fullsized":"2024-04-26T20:21:50.888239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:47.851667+0000","last_clean_scrub_stamp":"2024-04-26T20:21:47.851667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:18.679704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:21:59.926 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T20:21:59.926 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T20:21:59.926 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T20:21:59.926 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T20:22:00.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.061+0000 7f3bed35f640 1 Processor -- start 2024-04-26T20:22:00.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.061+0000 7f3bed35f640 1 -- start start 2024-04-26T20:22:00.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.062+0000 7f3bed35f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 0x7f3be80d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.062+0000 7f3bed35f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3be81673b0 0x7f3be81697a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.062+0000 7f3bed35f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3be815f940 con 0x7f3be815f560 2024-04-26T20:22:00.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.062+0000 7f3bed35f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3be80d2650 con 0x7f3be81673b0 2024-04-26T20:22:00.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.062+0000 7f3bed35f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3be80d1270 con 0x7f3be80d1c90 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.062+0000 7f3be77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3be81673b0 0x7f3be81697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.062+0000 7f3be77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3be81673b0 0x7f3be81697a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35250/0 (socket says 172.21.15.37:35250) 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.062+0000 7f3be77fe640 1 -- 172.21.15.37:0/2100697172 learned_addr learned my addr 172.21.15.37:0/2100697172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.062+0000 7f3be67fc640 1 --2- 172.21.15.37:0/2100697172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 0x7f3be80d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.063+0000 7f3be77fe640 1 -- 172.21.15.37:0/2100697172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 msgr2=0x7f3be80d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.063+0000 7f3be77fe640 1 --2- 172.21.15.37:0/2100697172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 0x7f3be80d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.063+0000 7f3be77fe640 1 -- 172.21.15.37:0/2100697172 >> v1:172.21.15.134:6789/0 conn(0x7f3be815f560 legacy=0x7f3be80d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.063+0000 7f3be77fe640 1 -- 172.21.15.37:0/2100697172 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be80d43a0 con 0x7f3be81673b0 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.063+0000 7f3be67fc640 1 --2- 172.21.15.37:0/2100697172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 0x7f3be80d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.063+0000 7f3be77fe640 1 --2- 172.21.15.37:0/2100697172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be81673b0 0x7f3be81697a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009870 tx=0x7f3be002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9707b5551a075008 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.063+0000 7f3be5ffb640 1 -- 172.21.15.37:0/2100697172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0037070 con 0x7f3be81673b0 2024-04-26T20:22:00.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.064+0000 7f3be5ffb640 1 -- 172.21.15.37:0/2100697172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be0032440 con 0x7f3be81673b0 2024-04-26T20:22:00.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.064+0000 7f3be5ffb640 1 -- 172.21.15.37:0/2100697172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0032810 con 0x7f3be81673b0 2024-04-26T20:22:00.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.064+0000 7f3bed35f640 1 -- 172.21.15.37:0/2100697172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be81673b0 msgr2=0x7f3be81697a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.064+0000 7f3bed35f640 1 --2- 172.21.15.37:0/2100697172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be81673b0 0x7f3be81697a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009870 tx=0x7f3be002a270 comp rx=0 tx=0).stop 2024-04-26T20:22:00.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.064+0000 7f3bed35f640 1 -- 172.21.15.37:0/2100697172 shutdown_connections 2024-04-26T20:22:00.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.064+0000 7f3bed35f640 1 --2- 172.21.15.37:0/2100697172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be81673b0 0x7f3be81697a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.064+0000 7f3bed35f640 1 --2- 172.21.15.37:0/2100697172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 0x7f3be80d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.064+0000 7f3bed35f640 1 -- 172.21.15.37:0/2100697172 >> 172.21.15.37:0/2100697172 conn(0x7f3be809e570 msgr2=0x7f3be80d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:00.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.064+0000 7f3bed35f640 1 -- 172.21.15.37:0/2100697172 shutdown_connections 2024-04-26T20:22:00.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.064+0000 7f3bed35f640 1 -- 172.21.15.37:0/2100697172 wait complete. 2024-04-26T20:22:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.065+0000 7f3bed35f640 1 Processor -- start 2024-04-26T20:22:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.065+0000 7f3bed35f640 1 -- start start 2024-04-26T20:22:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3bed35f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 0x7f3be81591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3bed35f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be815f560 0x7f3be80b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3bed35f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be81673b0 0x7f3be80b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3bed35f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3be80d38c0 con 0x7f3be815f560 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3bed35f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3be80d35e0 con 0x7f3be81673b0 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3bed35f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3be80d3ba0 con 0x7f3be80d1c90 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3be67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be815f560 0x7f3be80b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3be67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be815f560 0x7f3be80b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35260/0 (socket says 172.21.15.37:35260) 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3be67fc640 1 -- 172.21.15.37:0/3236805804 learned_addr learned my addr 172.21.15.37:0/3236805804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3be6ffd640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 0x7f3be81591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3be77fe640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be81673b0 0x7f3be80b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3be67fc640 1 -- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 msgr2=0x7f3be81591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3be67fc640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 0x7f3be81591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3be67fc640 1 -- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be81673b0 msgr2=0x7f3be80b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3be67fc640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be81673b0 0x7f3be80b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.066+0000 7f3be67fc640 1 -- 172.21.15.37:0/3236805804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be8159a70 con 0x7f3be815f560 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.067+0000 7f3be6ffd640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 0x7f3be81591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.067+0000 7f3be77fe640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be81673b0 0x7f3be80b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.067+0000 7f3be67fc640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be815f560 0x7f3be80b3bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000d7d0 tx=0x7f3bd000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.067+0000 7f3bdf7fe640 1 -- 172.21.15.37:0/3236805804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0004030 con 0x7f3be815f560 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.067+0000 7f3bdf7fe640 1 -- 172.21.15.37:0/3236805804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd00041d0 con 0x7f3be815f560 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.067+0000 7f3bed35f640 1 -- 172.21.15.37:0/3236805804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be80ce910 con 0x7f3be815f560 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.067+0000 7f3bdf7fe640 1 -- 172.21.15.37:0/3236805804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00096a0 con 0x7f3be815f560 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.067+0000 7f3bed35f640 1 -- 172.21.15.37:0/3236805804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be80ceed0 con 0x7f3be815f560 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.068+0000 7f3bed35f640 1 -- 172.21.15.37:0/3236805804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be806a360 con 0x7f3be815f560 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.068+0000 7f3bdf7fe640 1 -- 172.21.15.37:0/3236805804 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3bd0009840 con 0x7f3be815f560 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.069+0000 7f3bdf7fe640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bc806c8f0 0x7f3bc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.069+0000 7f3bdf7fe640 1 -- 172.21.15.37:0/3236805804 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3bd008bd40 con 0x7f3be815f560 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.069+0000 7f3be6ffd640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bc806c8f0 0x7f3bc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.069+0000 7f3be6ffd640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bc806c8f0 0x7f3bc806edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3be815c870 tx=0x7f3bd8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:00.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.073+0000 7f3bdf7fe640 1 -- 172.21.15.37:0/3236805804 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3bd005a200 con 0x7f3be815f560 2024-04-26T20:22:00.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.302+0000 7f3bed35f640 1 -- 172.21.15.37:0/3236805804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3be8162960 con 0x7f3be815f560 2024-04-26T20:22:00.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.302+0000 7f3bdf7fe640 1 -- 172.21.15.37:0/3236805804 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3bd0059b00 con 0x7f3be815f560 2024-04-26T20:22:00.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:00.303 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T20:22:00.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 -- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bc806c8f0 msgr2=0x7f3bc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bc806c8f0 0x7f3bc806edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3be815c870 tx=0x7f3bd8009040 comp rx=0 tx=0).stop 2024-04-26T20:22:00.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 -- 172.21.15.37:0/3236805804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be815f560 msgr2=0x7f3be80b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be815f560 0x7f3be80b3bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000d7d0 tx=0x7f3bd000dc90 comp rx=0 tx=0).stop 2024-04-26T20:22:00.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 -- 172.21.15.37:0/3236805804 shutdown_connections 2024-04-26T20:22:00.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be81673b0 0x7f3be80b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be815f560 0x7f3be80b3bf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3bc806c8f0 0x7f3bc806edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 --2- 172.21.15.37:0/3236805804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3be80d1c90 0x7f3be81591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 -- 172.21.15.37:0/3236805804 >> 172.21.15.37:0/3236805804 conn(0x7f3be809e570 msgr2=0x7f3be80b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:00.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 -- 172.21.15.37:0/3236805804 shutdown_connections 2024-04-26T20:22:00.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.305+0000 7f3bed35f640 1 -- 172.21.15.37:0/3236805804 wait complete. 2024-04-26T20:22:00.316 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T20:22:00.316 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.416+0000 7ff8201c8640 1 Processor -- start 2024-04-26T20:22:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.417+0000 7ff8201c8640 1 -- start start 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.417+0000 7ff8201c8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff818162e40 0x7ff8180d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.417+0000 7ff8201c8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8180cff40 0x7ff8180ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.417+0000 7ff8201c8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8180b1d70 con 0x7ff8180b2050 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.417+0000 7ff8201c8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff818163220 con 0x7ff818162e40 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.417+0000 7ff8201c8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff8180d0eb0 con 0x7ff8180cff40 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.417+0000 7ff81df64640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff818162e40 0x7ff8180d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.418+0000 7ff81df64640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff818162e40 0x7ff8180d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35264/0 (socket says 172.21.15.37:35264) 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.418+0000 7ff81df64640 1 -- 172.21.15.37:0/4272501902 learned_addr learned my addr 172.21.15.37:0/4272501902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.418+0000 7ff81e765640 1 --2- 172.21.15.37:0/4272501902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8180cff40 0x7ff8180ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.418+0000 7ff81df64640 1 -- 172.21.15.37:0/4272501902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8180cff40 msgr2=0x7ff8180ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.418+0000 7ff81df64640 1 --2- 172.21.15.37:0/4272501902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8180cff40 0x7ff8180ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.418+0000 7ff81df64640 1 -- 172.21.15.37:0/4272501902 >> v1:172.21.15.134:6789/0 conn(0x7ff8180b2050 legacy=0x7ff8180cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.418+0000 7ff81df64640 1 -- 172.21.15.37:0/4272501902 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8180d1960 con 0x7ff818162e40 2024-04-26T20:22:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.418+0000 7ff81e765640 1 --2- 172.21.15.37:0/4272501902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8180cff40 0x7ff8180ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:00.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.418+0000 7ff81df64640 1 --2- 172.21.15.37:0/4272501902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff818162e40 0x7ff8180d1340 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff818075e20 tx=0x7ff81402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b181d248be36dad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:00.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.419+0000 7ff81cf62640 1 -- 172.21.15.37:0/4272501902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814037070 con 0x7ff818162e40 2024-04-26T20:22:00.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.419+0000 7ff81cf62640 1 -- 172.21.15.37:0/4272501902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81402ae50 con 0x7ff818162e40 2024-04-26T20:22:00.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.419+0000 7ff81cf62640 1 -- 172.21.15.37:0/4272501902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814032660 con 0x7ff818162e40 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.419+0000 7ff8201c8640 1 -- 172.21.15.37:0/4272501902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff818162e40 msgr2=0x7ff8180d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.419+0000 7ff8201c8640 1 --2- 172.21.15.37:0/4272501902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff818162e40 0x7ff8180d1340 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff818075e20 tx=0x7ff81402a140 comp rx=0 tx=0).stop 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.419+0000 7ff8201c8640 1 -- 172.21.15.37:0/4272501902 shutdown_connections 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.419+0000 7ff8201c8640 1 --2- 172.21.15.37:0/4272501902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8180cff40 0x7ff8180ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.419+0000 7ff8201c8640 1 --2- 172.21.15.37:0/4272501902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff818162e40 0x7ff8180d1340 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.419+0000 7ff8201c8640 1 -- 172.21.15.37:0/4272501902 >> 172.21.15.37:0/4272501902 conn(0x7ff81809ba50 msgr2=0x7ff81809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.419+0000 7ff8201c8640 1 -- 172.21.15.37:0/4272501902 shutdown_connections 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.419+0000 7ff8201c8640 1 -- 172.21.15.37:0/4272501902 wait complete. 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.420+0000 7ff8201c8640 1 Processor -- start 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.420+0000 7ff8201c8640 1 -- start start 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.421+0000 7ff8201c8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8180b2050 0x7ff8180c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.421+0000 7ff8201c8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8180cff40 0x7ff8181f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.421+0000 7ff8201c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff818162e40 0x7ff8180c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.421+0000 7ff8201c8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8180b24e0 con 0x7ff818162e40 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.421+0000 7ff8201c8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8180d1f20 con 0x7ff8180cff40 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.421+0000 7ff8201c8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff81806ea40 con 0x7ff8180b2050 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.421+0000 7ff81e765640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff818162e40 0x7ff8180c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.421+0000 7ff81e765640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff818162e40 0x7ff8180c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35278/0 (socket says 172.21.15.37:35278) 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.421+0000 7ff81e765640 1 -- 172.21.15.37:0/321944583 learned_addr learned my addr 172.21.15.37:0/321944583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.421+0000 7ff81df64640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8180b2050 0x7ff8180c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.421+0000 7ff81d763640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8180cff40 0x7ff8181f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff81e765640 1 -- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8180b2050 msgr2=0x7ff8180c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff81e765640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8180b2050 0x7ff8180c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff81e765640 1 -- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8180cff40 msgr2=0x7ff8181f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff81e765640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8180cff40 0x7ff8181f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff81e765640 1 -- 172.21.15.37:0/321944583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8180c6ff0 con 0x7ff818162e40 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff81d763640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8180cff40 0x7ff8181f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff81e765640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff818162e40 0x7ff8180c83d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff81000cf20 tx=0x7ff81000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff80e7fc640 1 -- 172.21.15.37:0/321944583 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81001a070 con 0x7ff818162e40 2024-04-26T20:22:00.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff80e7fc640 1 -- 172.21.15.37:0/321944583 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81000d840 con 0x7ff818162e40 2024-04-26T20:22:00.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff80e7fc640 1 -- 172.21.15.37:0/321944583 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81000dc30 con 0x7ff818162e40 2024-04-26T20:22:00.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff8201c8640 1 -- 172.21.15.37:0/321944583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8180c72d0 con 0x7ff818162e40 2024-04-26T20:22:00.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.422+0000 7ff8201c8640 1 -- 172.21.15.37:0/321944583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8180c7890 con 0x7ff818162e40 2024-04-26T20:22:00.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.423+0000 7ff8201c8640 1 -- 172.21.15.37:0/321944583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81806a360 con 0x7ff818162e40 2024-04-26T20:22:00.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.423+0000 7ff80e7fc640 1 -- 172.21.15.37:0/321944583 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff81001d070 con 0x7ff818162e40 2024-04-26T20:22:00.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.424+0000 7ff80e7fc640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff7fc06c8f0 0x7ff7fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.424+0000 7ff80e7fc640 1 -- 172.21.15.37:0/321944583 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff81008c360 con 0x7ff818162e40 2024-04-26T20:22:00.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.424+0000 7ff81df64640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff7fc06c8f0 0x7ff7fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.424+0000 7ff81df64640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff7fc06c8f0 0x7ff7fc06edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff814034000 tx=0x7ff81402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:00.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.428+0000 7ff80e7fc640 1 -- 172.21.15.37:0/321944583 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff81005a140 con 0x7ff818162e40 2024-04-26T20:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.616+0000 7ff8201c8640 1 -- 172.21.15.37:0/321944583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff81809bfd0 con 0x7ff818162e40 2024-04-26T20:22:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.617+0000 7ff80e7fc640 1 -- 172.21.15.37:0/321944583 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7ff81005e440 con 0x7ff818162e40 2024-04-26T20:22:00.617 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:00.617 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 -- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff7fc06c8f0 msgr2=0x7ff7fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff7fc06c8f0 0x7ff7fc06edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff814034000 tx=0x7ff81402e000 comp rx=0 tx=0).stop 2024-04-26T20:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 -- 172.21.15.37:0/321944583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff818162e40 msgr2=0x7ff8180c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff818162e40 0x7ff8180c83d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff81000cf20 tx=0x7ff81000a7d0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 -- 172.21.15.37:0/321944583 shutdown_connections 2024-04-26T20:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff818162e40 0x7ff8180c83d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8180cff40 0x7ff8181f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff7fc06c8f0 0x7ff7fc06edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 --2- 172.21.15.37:0/321944583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8180b2050 0x7ff8180c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 -- 172.21.15.37:0/321944583 >> 172.21.15.37:0/321944583 conn(0x7ff81809ba50 msgr2=0x7ff8180b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 -- 172.21.15.37:0/321944583 shutdown_connections 2024-04-26T20:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.619+0000 7ff8201c8640 1 -- 172.21.15.37:0/321944583 wait complete. 2024-04-26T20:22:00.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:00.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:00.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.733+0000 7eff07a86640 1 Processor -- start 2024-04-26T20:22:00.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.734+0000 7eff07a86640 1 -- start start 2024-04-26T20:22:00.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.734+0000 7eff07a86640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7eff0015f400 0x7eff000b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.734+0000 7eff07a86640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff000b2760 con 0x7eff000d5a80 2024-04-26T20:22:00.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.734+0000 7eff07a86640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff000b2a00 con 0x7eff000b4cf0 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.734+0000 7eff07a86640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7eff000b43e0 con 0x7eff0015f400 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff05021640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7eff000d5a80 0x7eff000b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44390/0 (socket says 172.21.15.37:44390) 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff05021640 1 -- 172.21.15.37:0/2471310341 learned_addr learned my addr 172.21.15.37:0/2471310341 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 950660728 0 0) 0x7eff000b2760 con 0x7eff000d5a80 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff05822640 1 --2- 172.21.15.37:0/2471310341 >> v2:172.21.15.134:3300/0 conn(0x7eff0015f400 0x7eff000b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeec003520 con 0x7eff000d5a80 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3433450108 0 0) 0x7efeec003520 con 0x7eff000d5a80 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 >> v2:172.21.15.134:3300/0 conn(0x7eff0015f400 msgr2=0x7eff000b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 --2- 172.21.15.37:0/2471310341 >> v2:172.21.15.134:3300/0 conn(0x7eff0015f400 0x7eff000b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 >> v1:172.21.15.134:6789/0 conn(0x7eff000b4cf0 legacy=0x7eff001630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff000b2ce0 con 0x7eff000d5a80 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7efef0001690 con 0x7eff000d5a80 2024-04-26T20:22:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 >> v1:172.21.15.37:6789/0 conn(0x7eff000d5a80 legacy=0x7eff000b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 --2- 172.21.15.37:0/2471310341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec0039b0 0x7efeec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 --2- 172.21.15.37:0/2471310341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec004800 0x7efeec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 --2- 172.21.15.37:0/2471310341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeec0051c0 0x7efeec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efeec00c130 con 0x7efeec004800 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efeec00c2e0 con 0x7efeec0039b0 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efeec00c4e0 con 0x7efeec0051c0 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff05021640 1 --2- 172.21.15.37:0/2471310341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec004800 0x7efeec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff05822640 1 --2- 172.21.15.37:0/2471310341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec0039b0 0x7efeec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.735+0000 7eff06023640 1 --2- 172.21.15.37:0/2471310341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeec0051c0 0x7efeec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.736+0000 7eff05021640 1 -- 172.21.15.37:0/2471310341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeec0051c0 msgr2=0x7efeec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.736+0000 7eff05021640 1 --2- 172.21.15.37:0/2471310341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeec0051c0 0x7efeec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.736+0000 7eff05021640 1 -- 172.21.15.37:0/2471310341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec0039b0 msgr2=0x7efeec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.736+0000 7eff05021640 1 --2- 172.21.15.37:0/2471310341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec0039b0 0x7efeec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.736+0000 7eff05021640 1 -- 172.21.15.37:0/2471310341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec003520 con 0x7efeec004800 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.736+0000 7eff06023640 1 -- 172.21.15.37:0/2471310341 reap_dead start 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.736+0000 7eff05021640 1 --2- 172.21.15.37:0/2471310341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec004800 0x7efeec004c80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efef000c210 tx=0x7efeec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12be4a584b0d1bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.736+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0007d30 con 0x7efeec004800 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.736+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef0005ad0 con 0x7efeec004800 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.736+0000 7eff04820640 1 -- 172.21.15.37:0/2471310341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0007070 con 0x7efeec004800 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.737+0000 7eff07a86640 1 -- 172.21.15.37:0/2471310341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec004800 msgr2=0x7efeec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.737+0000 7eff07a86640 1 --2- 172.21.15.37:0/2471310341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec004800 0x7efeec004c80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efef000c210 tx=0x7efeec005830 comp rx=0 tx=0).stop 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.737+0000 7eff07a86640 1 -- 172.21.15.37:0/2471310341 shutdown_connections 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.737+0000 7eff07a86640 1 --2- 172.21.15.37:0/2471310341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec004800 0x7efeec004c80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.737+0000 7eff07a86640 1 -- 172.21.15.37:0/2471310341 >> 172.21.15.37:0/2471310341 conn(0x7eff0009e4f0 msgr2=0x7eff000d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.737+0000 7eff07a86640 1 -- 172.21.15.37:0/2471310341 shutdown_connections 2024-04-26T20:22:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.737+0000 7eff07a86640 1 -- 172.21.15.37:0/2471310341 wait complete. 2024-04-26T20:22:00.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.738+0000 7eff07a86640 1 Processor -- start 2024-04-26T20:22:00.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.738+0000 7eff07a86640 1 -- start start 2024-04-26T20:22:00.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff07a86640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeec004800 0x7eff000cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff07a86640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff0015cd90 0x7eff0015d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff07a86640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff001f4bd0 0x7eff001f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff07a86640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff0015f7e0 con 0x7eff0015cd90 2024-04-26T20:22:00.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff07a86640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff000b32a0 con 0x7eff001f4bd0 2024-04-26T20:22:00.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff07a86640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7eff0015f960 con 0x7efeec004800 2024-04-26T20:22:00.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff05021640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff0015cd90 0x7eff0015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff05021640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff0015cd90 0x7eff0015d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35284/0 (socket says 172.21.15.37:35284) 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff05021640 1 -- 172.21.15.37:0/3076917038 learned_addr learned my addr 172.21.15.37:0/3076917038 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff05822640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeec004800 0x7eff000cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff06023640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff001f4bd0 0x7eff001f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff05021640 1 -- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeec004800 msgr2=0x7eff000cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff05021640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeec004800 0x7eff000cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff05021640 1 -- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff001f4bd0 msgr2=0x7eff001f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff05021640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff001f4bd0 0x7eff001f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff05021640 1 -- 172.21.15.37:0/3076917038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0015d750 con 0x7eff0015cd90 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.739+0000 7eff05021640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff0015cd90 0x7eff0015d210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efef000c210 tx=0x7eff000b2bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.740+0000 7efef67fc640 1 -- 172.21.15.37:0/3076917038 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef003f020 con 0x7eff0015cd90 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.740+0000 7efef67fc640 1 -- 172.21.15.37:0/3076917038 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef0001450 con 0x7eff0015cd90 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.740+0000 7eff07a86640 1 -- 172.21.15.37:0/3076917038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff000b8970 con 0x7eff0015cd90 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.740+0000 7efef67fc640 1 -- 172.21.15.37:0/3076917038 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef00047e0 con 0x7eff0015cd90 2024-04-26T20:22:00.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.740+0000 7eff07a86640 1 -- 172.21.15.37:0/3076917038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff000b8f30 con 0x7eff0015cd90 2024-04-26T20:22:00.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.741+0000 7eff07a86640 1 -- 172.21.15.37:0/3076917038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0006a360 con 0x7eff0015cd90 2024-04-26T20:22:00.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.741+0000 7efef67fc640 1 -- 172.21.15.37:0/3076917038 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efef0042020 con 0x7eff0015cd90 2024-04-26T20:22:00.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.742+0000 7efef67fc640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efee406c8f0 0x7efee406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:00.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.742+0000 7efef67fc640 1 -- 172.21.15.37:0/3076917038 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efef00ab6d0 con 0x7eff0015cd90 2024-04-26T20:22:00.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.742+0000 7eff05822640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efee406c8f0 0x7efee406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:00.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.742+0000 7eff05822640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efee406c8f0 0x7efee406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff000d44e0 tx=0x7efefc009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:00.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.746+0000 7efef67fc640 1 -- 172.21.15.37:0/3076917038 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efef0079b20 con 0x7eff0015cd90 2024-04-26T20:22:00.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.931+0000 7eff07a86640 1 -- 172.21.15.37:0/3076917038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff000d28f0 con 0x7eff0015cd90 2024-04-26T20:22:00.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.931+0000 7efef67fc640 1 -- 172.21.15.37:0/3076917038 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7efef0079420 con 0x7eff0015cd90 2024-04-26T20:22:00.932 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 -- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efee406c8f0 msgr2=0x7efee406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efee406c8f0 0x7efee406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff000d44e0 tx=0x7efefc009190 comp rx=0 tx=0).stop 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 -- 172.21.15.37:0/3076917038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff0015cd90 msgr2=0x7eff0015d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff0015cd90 0x7eff0015d210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efef000c210 tx=0x7eff000b2bd0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 -- 172.21.15.37:0/3076917038 shutdown_connections 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff001f4bd0 0x7eff001f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff0015cd90 0x7eff0015d210 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efee406c8f0 0x7efee406edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 --2- 172.21.15.37:0/3076917038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeec004800 0x7eff000cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 -- 172.21.15.37:0/3076917038 >> 172.21.15.37:0/3076917038 conn(0x7eff0009e4f0 msgr2=0x7eff000d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 -- 172.21.15.37:0/3076917038 shutdown_connections 2024-04-26T20:22:00.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:00.933+0000 7eff07a86640 1 -- 172.21.15.37:0/3076917038 wait complete. 2024-04-26T20:22:00.945 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.048+0000 7fbd0b674640 1 Processor -- start 2024-04-26T20:22:01.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.049+0000 7fbd0b674640 1 -- start start 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.049+0000 7fbd0b674640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 0x7fbd0416a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.049+0000 7fbd0b674640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd040b2760 con 0x7fbd040d5a80 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.049+0000 7fbd0b674640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd040b2a00 con 0x7fbd0415f400 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.049+0000 7fbd0b674640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbd040b43e0 con 0x7fbd040b4cf0 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.050+0000 7fbd08c0f640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbd040d5a80 0x7fbd040b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44396/0 (socket says 172.21.15.37:44396) 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.050+0000 7fbd08c0f640 1 -- 172.21.15.37:0/2053680361 learned_addr learned my addr 172.21.15.37:0/2053680361 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.050+0000 7fbd09c11640 1 --2- 172.21.15.37:0/2053680361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 0x7fbd0416a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.050+0000 7fbcfbfff640 1 -- 172.21.15.37:0/2053680361 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1281986436 0 0) 0x7fbd040b2760 con 0x7fbd040d5a80 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.050+0000 7fbcfbfff640 1 -- 172.21.15.37:0/2053680361 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcf0003520 con 0x7fbd040d5a80 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.050+0000 7fbcfbfff640 1 -- 172.21.15.37:0/2053680361 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 401102504 0 0) 0x7fbd040b2a00 con 0x7fbd0415f400 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.050+0000 7fbcfbfff640 1 -- 172.21.15.37:0/2053680361 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd040b2760 con 0x7fbd0415f400 2024-04-26T20:22:01.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.050+0000 7fbd09c11640 1 -- 172.21.15.37:0/2053680361 >> v1:172.21.15.134:6789/0 conn(0x7fbd0415f400 legacy=0x7fbd040b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.050+0000 7fbcfbfff640 1 -- 172.21.15.37:0/2053680361 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1496538140 0 0) 0x7fbcf0003520 con 0x7fbd040d5a80 2024-04-26T20:22:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.050+0000 7fbd09c11640 1 -- 172.21.15.37:0/2053680361 >> v1:172.21.15.37:6789/0 conn(0x7fbd040d5a80 legacy=0x7fbd040b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.050+0000 7fbd09c11640 1 -- 172.21.15.37:0/2053680361 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd040b2ce0 con 0x7fbd040b4cf0 2024-04-26T20:22:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.051+0000 7fbd09c11640 1 --2- 172.21.15.37:0/2053680361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 0x7fbd0416a3e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009dc0 tx=0x7fbcfc02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d316ea23be73836c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.051+0000 7fbcfbfff640 1 -- 172.21.15.37:0/2053680361 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4001670 con 0x7fbd040b4cf0 2024-04-26T20:22:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.051+0000 7fbcfbfff640 1 -- 172.21.15.37:0/2053680361 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcfc007540 con 0x7fbd040b4cf0 2024-04-26T20:22:01.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.051+0000 7fbcfbfff640 1 -- 172.21.15.37:0/2053680361 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc0078f0 con 0x7fbd040b4cf0 2024-04-26T20:22:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.052+0000 7fbd0b674640 1 -- 172.21.15.37:0/2053680361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 msgr2=0x7fbd0416a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.052+0000 7fbd0b674640 1 --2- 172.21.15.37:0/2053680361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 0x7fbd0416a3e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009dc0 tx=0x7fbcfc02a5d0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.052+0000 7fbd0b674640 1 -- 172.21.15.37:0/2053680361 shutdown_connections 2024-04-26T20:22:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.052+0000 7fbd0b674640 1 --2- 172.21.15.37:0/2053680361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 0x7fbd0416a3e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.052+0000 7fbd0b674640 1 -- 172.21.15.37:0/2053680361 >> 172.21.15.37:0/2053680361 conn(0x7fbd0409e4f0 msgr2=0x7fbd040d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.052+0000 7fbd0b674640 1 -- 172.21.15.37:0/2053680361 shutdown_connections 2024-04-26T20:22:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.052+0000 7fbd0b674640 1 -- 172.21.15.37:0/2053680361 wait complete. 2024-04-26T20:22:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.053+0000 7fbd0b674640 1 Processor -- start 2024-04-26T20:22:01.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.053+0000 7fbd0b674640 1 -- start start 2024-04-26T20:22:01.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd0b674640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 0x7fbd040cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd0b674640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd040d5a80 0x7fbd040cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd0b674640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0415f400 0x7fbd0414d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd0b674640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd040b32a0 con 0x7fbd0415f400 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd0b674640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd040b3ed0 con 0x7fbd040d5a80 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd0b674640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbd0415f7e0 con 0x7fbd040b4cf0 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd09c11640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0415f400 0x7fbd0414d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd09c11640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0415f400 0x7fbd0414d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35300/0 (socket says 172.21.15.37:35300) 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd09c11640 1 -- 172.21.15.37:0/1271742364 learned_addr learned my addr 172.21.15.37:0/1271742364 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd09410640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 0x7fbd040cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd08c0f640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd040d5a80 0x7fbd040cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd09c11640 1 -- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 msgr2=0x7fbd040cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd09c11640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 0x7fbd040cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd09c11640 1 -- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd040d5a80 msgr2=0x7fbd040cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd09c11640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd040d5a80 0x7fbd040cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd09c11640 1 -- 172.21.15.37:0/1271742364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0414c590 con 0x7fbd0415f400 2024-04-26T20:22:01.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd09410640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 0x7fbd040cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd09c11640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0415f400 0x7fbd0414d630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009ee0 tx=0x7fbcfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbcf9ffb640 1 -- 172.21.15.37:0/1271742364 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc03f070 con 0x7fbd0415f400 2024-04-26T20:22:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbcf9ffb640 1 -- 172.21.15.37:0/1271742364 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcfc031c40 con 0x7fbd0415f400 2024-04-26T20:22:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbd0b674640 1 -- 172.21.15.37:0/1271742364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0414c870 con 0x7fbd0415f400 2024-04-26T20:22:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.054+0000 7fbcf9ffb640 1 -- 172.21.15.37:0/1271742364 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc009450 con 0x7fbd0415f400 2024-04-26T20:22:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.055+0000 7fbd0b674640 1 -- 172.21.15.37:0/1271742364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0414ce30 con 0x7fbd0415f400 2024-04-26T20:22:01.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.056+0000 7fbd0b674640 1 -- 172.21.15.37:0/1271742364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0406a360 con 0x7fbd0415f400 2024-04-26T20:22:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.056+0000 7fbcf9ffb640 1 -- 172.21.15.37:0/1271742364 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcfc043410 con 0x7fbd0415f400 2024-04-26T20:22:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.056+0000 7fbcf9ffb640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbce806c8f0 0x7fbce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.057+0000 7fbcf9ffb640 1 -- 172.21.15.37:0/1271742364 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbcfc0ace00 con 0x7fbd0415f400 2024-04-26T20:22:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.057+0000 7fbd09410640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbce806c8f0 0x7fbce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.057+0000 7fbd09410640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbce806c8f0 0x7fbce806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbd00000f60 tx=0x7fbd00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.061+0000 7fbcf9ffb640 1 -- 172.21.15.37:0/1271742364 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbcfc07b2c0 con 0x7fbd0415f400 2024-04-26T20:22:01.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.247+0000 7fbd0b674640 1 -- 172.21.15.37:0/1271742364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd04075fb0 con 0x7fbd0415f400 2024-04-26T20:22:01.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.247+0000 7fbcf9ffb640 1 -- 172.21.15.37:0/1271742364 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fbcfc07abc0 con 0x7fbd0415f400 2024-04-26T20:22:01.248 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:01.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 -- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbce806c8f0 msgr2=0x7fbce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbce806c8f0 0x7fbce806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbd00000f60 tx=0x7fbd00009040 comp rx=0 tx=0).stop 2024-04-26T20:22:01.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 -- 172.21.15.37:0/1271742364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0415f400 msgr2=0x7fbd0414d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0415f400 0x7fbd0414d630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009ee0 tx=0x7fbcfc004a40 comp rx=0 tx=0).stop 2024-04-26T20:22:01.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 -- 172.21.15.37:0/1271742364 shutdown_connections 2024-04-26T20:22:01.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0415f400 0x7fbd0414d630 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd040d5a80 0x7fbd040cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbce806c8f0 0x7fbce806edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 --2- 172.21.15.37:0/1271742364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd040b4cf0 0x7fbd040cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 -- 172.21.15.37:0/1271742364 >> 172.21.15.37:0/1271742364 conn(0x7fbd0409e4f0 msgr2=0x7fbd040d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:01.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 -- 172.21.15.37:0/1271742364 shutdown_connections 2024-04-26T20:22:01.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.249+0000 7fbd0b674640 1 -- 172.21.15.37:0/1271742364 wait complete. 2024-04-26T20:22:01.260 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.363+0000 7f1ec9df7640 1 Processor -- start 2024-04-26T20:22:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.363+0000 7f1ec9df7640 1 -- start start 2024-04-26T20:22:01.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.364+0000 7f1ec9df7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1ec415cdf0 0x7f1ec40d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.364+0000 7f1ec9df7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec40d2590 con 0x7f1ec415d1d0 2024-04-26T20:22:01.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.364+0000 7f1ec9df7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec40d2870 con 0x7f1ec40d32d0 2024-04-26T20:22:01.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.364+0000 7f1ec9df7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec40d4310 con 0x7f1ec415cdf0 2024-04-26T20:22:01.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.364+0000 7f1ec3fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1ec415d1d0 0x7f1ec40b58b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44404/0 (socket says 172.21.15.37:44404) 2024-04-26T20:22:01.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.364+0000 7f1ec3fff640 1 -- 172.21.15.37:0/3200806553 learned_addr learned my addr 172.21.15.37:0/3200806553 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.364+0000 7f1ec37fe640 1 --2- 172.21.15.37:0/3200806553 >> v2:172.21.15.134:3300/0 conn(0x7f1ec415cdf0 0x7f1ec40d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.364+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 65876994 0 0) 0x7f1ec40d2590 con 0x7f1ec415d1d0 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1eac003520 con 0x7f1ec415d1d0 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 26880246 0 0) 0x7f1eac003520 con 0x7f1ec415d1d0 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 >> v2:172.21.15.134:3300/0 conn(0x7f1ec415cdf0 msgr2=0x7f1ec40d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 --2- 172.21.15.37:0/3200806553 >> v2:172.21.15.134:3300/0 conn(0x7f1ec415cdf0 0x7f1ec40d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 >> v1:172.21.15.134:6789/0 conn(0x7f1ec40d32d0 legacy=0x7f1ec40d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec40b50b0 con 0x7f1ec415d1d0 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f1ebc0019f0 con 0x7f1ec415d1d0 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec37fe640 1 --2- 172.21.15.37:0/3200806553 >> v2:172.21.15.134:3300/0 conn(0x7f1ec415cdf0 0x7f1ec40d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 >> v1:172.21.15.37:6789/0 conn(0x7f1ec415d1d0 legacy=0x7f1ec40b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac0039b0 0x7f1eac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 0x7f1eac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eac0051c0 0x7f1eac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac00c030 con 0x7f1eac004800 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac00c1e0 con 0x7f1eac0039b0 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1eac00c3e0 con 0x7f1eac0051c0 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec2ffd640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 0x7f1eac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec37fe640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac0039b0 0x7f1eac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec3fff640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eac0051c0 0x7f1eac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec2ffd640 1 -- 172.21.15.37:0/3200806553 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eac0051c0 msgr2=0x7f1eac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec2ffd640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eac0051c0 0x7f1eac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec2ffd640 1 -- 172.21.15.37:0/3200806553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac0039b0 msgr2=0x7f1eac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec2ffd640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac0039b0 0x7f1eac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec2ffd640 1 -- 172.21.15.37:0/3200806553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac003520 con 0x7f1eac004800 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec3fff640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eac0051c0 0x7f1eac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.365+0000 7f1ec3fff640 1 -- 172.21.15.37:0/3200806553 reap_dead start 2024-04-26T20:22:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.366+0000 7f1ec2ffd640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 0x7f1eac004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8001720 tx=0x7f1eac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25da9f60d0c11d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.366+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8001370 con 0x7f1eac004800 2024-04-26T20:22:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.366+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb80052d0 con 0x7f1eac004800 2024-04-26T20:22:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.366+0000 7f1ec27fc640 1 -- 172.21.15.37:0/3200806553 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb802ab90 con 0x7f1eac004800 2024-04-26T20:22:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.367+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3200806553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 msgr2=0x7f1eac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.367+0000 7f1ec9df7640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 0x7f1eac004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8001720 tx=0x7f1eac005830 comp rx=0 tx=0).stop 2024-04-26T20:22:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.367+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3200806553 shutdown_connections 2024-04-26T20:22:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.367+0000 7f1ec9df7640 1 --2- 172.21.15.37:0/3200806553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 0x7f1eac004c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.367+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3200806553 >> 172.21.15.37:0/3200806553 conn(0x7f1ec408ec30 msgr2=0x7f1ec409e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.367+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3200806553 shutdown_connections 2024-04-26T20:22:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.367+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3200806553 wait complete. 2024-04-26T20:22:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.368+0000 7f1ec9df7640 1 Processor -- start 2024-04-26T20:22:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.368+0000 7f1ec9df7640 1 -- start start 2024-04-26T20:22:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec9df7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 0x7f1ec4159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec9df7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec40ba960 0x7f1ec4159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec9df7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec40b7400 0x7f1ec40b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec9df7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec40d2d90 con 0x7f1eac004800 2024-04-26T20:22:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec9df7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec415fcc0 con 0x7f1ec40ba960 2024-04-26T20:22:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec9df7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec40d2f10 con 0x7f1ec40b7400 2024-04-26T20:22:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 0x7f1ec4159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 0x7f1ec4159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35320/0 (socket says 172.21.15.37:35320) 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec37fe640 1 -- 172.21.15.37:0/3690871820 learned_addr learned my addr 172.21.15.37:0/3690871820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec3fff640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec40b7400 0x7f1ec40b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec2ffd640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec40ba960 0x7f1ec4159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec37fe640 1 -- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec40b7400 msgr2=0x7f1ec40b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec37fe640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec40b7400 0x7f1ec40b63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec37fe640 1 -- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec40ba960 msgr2=0x7f1ec4159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec37fe640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec40ba960 0x7f1ec4159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec37fe640 1 -- 172.21.15.37:0/3690871820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec40b7890 con 0x7f1eac004800 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec2ffd640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec40ba960 0x7f1ec4159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1ec37fe640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 0x7f1ec4159b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4011d00 tx=0x7f1eb400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1eabfff640 1 -- 172.21.15.37:0/3690871820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400fd20 con 0x7f1eac004800 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.369+0000 7f1eabfff640 1 -- 172.21.15.37:0/3690871820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb4004020 con 0x7f1eac004800 2024-04-26T20:22:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.370+0000 7f1eabfff640 1 -- 172.21.15.37:0/3690871820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4004410 con 0x7f1eac004800 2024-04-26T20:22:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.370+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3690871820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec40b58e0 con 0x7f1eac004800 2024-04-26T20:22:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.370+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3690871820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec40b5ea0 con 0x7f1eac004800 2024-04-26T20:22:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.371+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3690871820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec406a360 con 0x7f1eac004800 2024-04-26T20:22:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.371+0000 7f1eabfff640 1 -- 172.21.15.37:0/3690871820 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1eb4012040 con 0x7f1eac004800 2024-04-26T20:22:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.371+0000 7f1eabfff640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb006c8f0 0x7f1eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.372+0000 7f1eabfff640 1 -- 172.21.15.37:0/3690871820 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1eb408c050 con 0x7f1eac004800 2024-04-26T20:22:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.372+0000 7f1ec2ffd640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb006c8f0 0x7f1eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.372+0000 7f1ec2ffd640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb006c8f0 0x7f1eb006edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8032000 tx=0x7f1eb802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.376+0000 7f1eabfff640 1 -- 172.21.15.37:0/3690871820 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1eb4019030 con 0x7f1eac004800 2024-04-26T20:22:01.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.562+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3690871820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1ec4160900 con 0x7f1eac004800 2024-04-26T20:22:01.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.562+0000 7f1eabfff640 1 -- 172.21.15.37:0/3690871820 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1eb4059ec0 con 0x7f1eac004800 2024-04-26T20:22:01.562 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.564+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb006c8f0 msgr2=0x7f1eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.564+0000 7f1ec9df7640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb006c8f0 0x7f1eb006edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8032000 tx=0x7f1eb802e000 comp rx=0 tx=0).stop 2024-04-26T20:22:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.564+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3690871820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 msgr2=0x7f1ec4159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.564+0000 7f1ec9df7640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 0x7f1ec4159b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4011d00 tx=0x7f1eb400f610 comp rx=0 tx=0).stop 2024-04-26T20:22:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.564+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3690871820 shutdown_connections 2024-04-26T20:22:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.564+0000 7f1ec9df7640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb006c8f0 0x7f1eb006edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.564+0000 7f1ec9df7640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec40b7400 0x7f1ec40b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.564+0000 7f1ec9df7640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec40ba960 0x7f1ec4159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.564+0000 7f1ec9df7640 1 --2- 172.21.15.37:0/3690871820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eac004800 0x7f1ec4159b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.564+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3690871820 >> 172.21.15.37:0/3690871820 conn(0x7f1ec408ec30 msgr2=0x7f1ec4161db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:01.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.565+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3690871820 shutdown_connections 2024-04-26T20:22:01.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.565+0000 7f1ec9df7640 1 -- 172.21.15.37:0/3690871820 wait complete. 2024-04-26T20:22:01.575 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:22:01.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.676+0000 7f0641664640 1 Processor -- start 2024-04-26T20:22:01.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.676+0000 7f0641664640 1 -- start start 2024-04-26T20:22:01.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f0641664640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f063c162b40 0x7f063c0d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f0641664640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f063c0d3820 0x7f063c0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f0641664640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f063c0d0210 con 0x7f063c0d1850 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f0641664640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f063c0d41c0 con 0x7f063c0d3820 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f0641664640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f063c0d04f0 con 0x7f063c162b40 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f063a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f063c0d3820 0x7f063c0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f063b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f063c0d1850 0x7f063c0d1ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44412/0 (socket says 172.21.15.37:44412) 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f063b7fe640 1 -- 172.21.15.37:0/320883875 learned_addr learned my addr 172.21.15.37:0/320883875 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f063affd640 1 --2- 172.21.15.37:0/320883875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f063c162b40 0x7f063c0d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f063a7fc640 1 -- 172.21.15.37:0/320883875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f063c162b40 msgr2=0x7f063c0d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f063a7fc640 1 --2- 172.21.15.37:0/320883875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f063c162b40 0x7f063c0d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f063a7fc640 1 -- 172.21.15.37:0/320883875 >> v1:172.21.15.37:6789/0 conn(0x7f063c0d1850 legacy=0x7f063c0d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f063a7fc640 1 -- 172.21.15.37:0/320883875 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c0d07d0 con 0x7f063c0d3820 2024-04-26T20:22:01.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.677+0000 7f0639ffb640 1 -- 172.21.15.37:0/320883875 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038754100 0 0) 0x7f063c0d0210 con 0x7f063c0d1850 2024-04-26T20:22:01.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.678+0000 7f063a7fc640 1 --2- 172.21.15.37:0/320883875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f063c0d3820 0x7f063c0d3c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06240097a0 tx=0x7f062402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5a659f7e23c76e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:01.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.678+0000 7f0639ffb640 1 -- 172.21.15.37:0/320883875 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624037070 con 0x7f063c0d3820 2024-04-26T20:22:01.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.678+0000 7f0639ffb640 1 -- 172.21.15.37:0/320883875 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0624032440 con 0x7f063c0d3820 2024-04-26T20:22:01.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.678+0000 7f0639ffb640 1 -- 172.21.15.37:0/320883875 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624032810 con 0x7f063c0d3820 2024-04-26T20:22:01.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.679+0000 7f0641664640 1 -- 172.21.15.37:0/320883875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f063c0d3820 msgr2=0x7f063c0d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.679+0000 7f0641664640 1 --2- 172.21.15.37:0/320883875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f063c0d3820 0x7f063c0d3c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06240097a0 tx=0x7f062402a210 comp rx=0 tx=0).stop 2024-04-26T20:22:01.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.679+0000 7f0641664640 1 -- 172.21.15.37:0/320883875 shutdown_connections 2024-04-26T20:22:01.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.679+0000 7f0641664640 1 --2- 172.21.15.37:0/320883875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f063c0d3820 0x7f063c0d3c80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.679+0000 7f0641664640 1 --2- 172.21.15.37:0/320883875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f063c162b40 0x7f063c0d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.679+0000 7f0641664640 1 -- 172.21.15.37:0/320883875 >> 172.21.15.37:0/320883875 conn(0x7f063c09e550 msgr2=0x7f063c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:01.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.679+0000 7f0641664640 1 -- 172.21.15.37:0/320883875 shutdown_connections 2024-04-26T20:22:01.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.679+0000 7f0641664640 1 -- 172.21.15.37:0/320883875 wait complete. 2024-04-26T20:22:01.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.680+0000 7f0641664640 1 Processor -- start 2024-04-26T20:22:01.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.680+0000 7f0641664640 1 -- start start 2024-04-26T20:22:01.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f0641664640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c0d1850 0x7f063c0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f0641664640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f063c0d3820 0x7f063c0ba220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f0641664640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f063c162b40 0x7f063c0ba760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f0641664640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f063c0b5bb0 con 0x7f063c0d3820 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f0641664640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f063c0d0d90 con 0x7f063c0d1850 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f0641664640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f063c0d6110 con 0x7f063c162b40 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f063c0d3820 0x7f063c0ba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f063c0d3820 0x7f063c0ba220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35332/0 (socket says 172.21.15.37:35332) 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063a7fc640 1 -- 172.21.15.37:0/2062542 learned_addr learned my addr 172.21.15.37:0/2062542 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063affd640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c0d1850 0x7f063c0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063b7fe640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f063c162b40 0x7f063c0ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063a7fc640 1 -- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f063c162b40 msgr2=0x7f063c0ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063a7fc640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f063c162b40 0x7f063c0ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063a7fc640 1 -- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c0d1850 msgr2=0x7f063c0cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063a7fc640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c0d1850 0x7f063c0cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063a7fc640 1 -- 172.21.15.37:0/2062542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c152ed0 con 0x7f063c0d3820 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063affd640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c0d1850 0x7f063c0cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:01.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.681+0000 7f063b7fe640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f063c162b40 0x7f063c0ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:01.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.682+0000 7f063a7fc640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f063c0d3820 0x7f063c0ba220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0624035040 tx=0x7f06240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:01.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.682+0000 7f06337fe640 1 -- 172.21.15.37:0/2062542 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624037040 con 0x7f063c0d3820 2024-04-26T20:22:01.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.682+0000 7f06337fe640 1 -- 172.21.15.37:0/2062542 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0624030500 con 0x7f063c0d3820 2024-04-26T20:22:01.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.682+0000 7f0641664640 1 -- 172.21.15.37:0/2062542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c06e270 con 0x7f063c0d3820 2024-04-26T20:22:01.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.682+0000 7f06337fe640 1 -- 172.21.15.37:0/2062542 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624030910 con 0x7f063c0d3820 2024-04-26T20:22:01.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.682+0000 7f0641664640 1 -- 172.21.15.37:0/2062542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c153470 con 0x7f063c0d3820 2024-04-26T20:22:01.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.683+0000 7f0641664640 1 -- 172.21.15.37:0/2062542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c06a360 con 0x7f063c0d3820 2024-04-26T20:22:01.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.683+0000 7f06337fe640 1 -- 172.21.15.37:0/2062542 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0624044020 con 0x7f063c0d3820 2024-04-26T20:22:01.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.683+0000 7f06337fe640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f061c06c920 0x7f061c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:01.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.684+0000 7f06337fe640 1 -- 172.21.15.37:0/2062542 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f06240abdb0 con 0x7f063c0d3820 2024-04-26T20:22:01.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.684+0000 7f063affd640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f061c06c920 0x7f061c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:01.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.684+0000 7f063affd640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f061c06c920 0x7f061c06ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0634009ac0 tx=0x7f0634008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:01.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.688+0000 7f06337fe640 1 -- 172.21.15.37:0/2062542 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f062407a1e0 con 0x7f063c0d3820 2024-04-26T20:22:01.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.911+0000 7f0641664640 1 -- 172.21.15.37:0/2062542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f063c0d2960 con 0x7f063c0d3820 2024-04-26T20:22:01.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.911+0000 7f06337fe640 1 -- 172.21.15.37:0/2062542 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7f0624079ae0 con 0x7f063c0d3820 2024-04-26T20:22:01.911 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:01.912 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2911741012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2911741012},{"type":"v1","addr":"172.21.15.134:6836","nonce":2911741012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1600483467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1600483467},{"type":"v1","addr":"172.21.15.37:6835","nonce":1600483467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2722614558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2722614558},{"type":"v1","addr":"172.21.15.134:6837","nonce":2722614558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:21:49.237855+0000","modified":"2024-04-26T20:21:52.255148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3920647125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3920647125},{"type":"v1","addr":"172.21.15.37:6834","nonce":3920647125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-04-26T20:22:01.912 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-26T20:22:01.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 -- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f061c06c920 msgr2=0x7f061c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f061c06c920 0x7f061c06ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0634009ac0 tx=0x7f0634008040 comp rx=0 tx=0).stop 2024-04-26T20:22:01.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 -- 172.21.15.37:0/2062542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f063c0d3820 msgr2=0x7f063c0ba220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:01.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f063c0d3820 0x7f063c0ba220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0624035040 tx=0x7f06240046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 -- 172.21.15.37:0/2062542 shutdown_connections 2024-04-26T20:22:01.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f061c06c920 0x7f061c06ede0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f063c162b40 0x7f063c0ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f063c0d3820 0x7f063c0ba220 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 --2- 172.21.15.37:0/2062542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c0d1850 0x7f063c0cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 -- 172.21.15.37:0/2062542 >> 172.21.15.37:0/2062542 conn(0x7f063c09e550 msgr2=0x7f063c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:01.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 -- 172.21.15.37:0/2062542 shutdown_connections 2024-04-26T20:22:01.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:01.913+0000 7f0641664640 1 -- 172.21.15.37:0/2062542 wait complete. 2024-04-26T20:22:01.925 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:02.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.026+0000 7f281efaf640 1 Processor -- start 2024-04-26T20:22:02.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.026+0000 7f281efaf640 1 -- start start 2024-04-26T20:22:02.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.026+0000 7f281efaf640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f281815f400 0x7f28180d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.026+0000 7f281efaf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28180d3220 con 0x7f28180d0b80 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.026+0000 7f281efaf640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28180d5ef0 con 0x7f28180d3e50 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.026+0000 7f281efaf640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f28180d26c0 con 0x7f281815f400 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f2817fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f28180d0b80 0x7f28180d3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44414/0 (socket says 172.21.15.37:44414) 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f2817fff640 1 -- 172.21.15.37:0/3853500749 learned_addr learned my addr 172.21.15.37:0/3853500749 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f281cd4b640 1 --2- 172.21.15.37:0/3853500749 >> v2:172.21.15.134:3300/0 conn(0x7f281815f400 0x7f28180d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2470021061 0 0) 0x7f28180d3220 con 0x7f28180d0b80 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2808003520 con 0x7f28180d0b80 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 557628013 0 0) 0x7f2808003520 con 0x7f28180d0b80 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 >> v2:172.21.15.134:3300/0 conn(0x7f281815f400 msgr2=0x7f28180d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 --2- 172.21.15.37:0/3853500749 >> v2:172.21.15.134:3300/0 conn(0x7f281815f400 0x7f28180d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 >> v1:172.21.15.134:6789/0 conn(0x7f28180d3e50 legacy=0x7f28180b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28180d00a0 con 0x7f28180d0b80 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f2804001670 con 0x7f28180d0b80 2024-04-26T20:22:02.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 >> v1:172.21.15.37:6789/0 conn(0x7f28180d0b80 legacy=0x7f28180d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 --2- 172.21.15.37:0/3853500749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28080039b0 0x7f2808003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 --2- 172.21.15.37:0/3853500749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2808004800 0x7f2808004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 --2- 172.21.15.37:0/3853500749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28080051c0 0x7f280800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f28180d26c0 con 0x7f28080051c0 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f280800c060 con 0x7f2808004800 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f280800c260 con 0x7f28080039b0 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f281d54c640 1 --2- 172.21.15.37:0/3853500749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28080051c0 0x7f280800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f2817fff640 1 --2- 172.21.15.37:0/3853500749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2808004800 0x7f2808004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.027+0000 7f281cd4b640 1 --2- 172.21.15.37:0/3853500749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28080039b0 0x7f2808003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.028+0000 7f281d54c640 1 -- 172.21.15.37:0/3853500749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28080039b0 msgr2=0x7f2808003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.028+0000 7f281d54c640 1 --2- 172.21.15.37:0/3853500749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28080039b0 0x7f2808003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.028+0000 7f281d54c640 1 -- 172.21.15.37:0/3853500749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2808004800 msgr2=0x7f2808004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.028+0000 7f281d54c640 1 --2- 172.21.15.37:0/3853500749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2808004800 0x7f2808004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.028+0000 7f281d54c640 1 -- 172.21.15.37:0/3853500749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2808003520 con 0x7f28080051c0 2024-04-26T20:22:02.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.028+0000 7f281d54c640 1 -- 172.21.15.37:0/3853500749 reap_dead start 2024-04-26T20:22:02.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.028+0000 7f281d54c640 1 --2- 172.21.15.37:0/3853500749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28080051c0 0x7f280800b9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2810001e20 tx=0x7f2808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c451f1552e319b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.028+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810001a70 con 0x7f28080051c0 2024-04-26T20:22:02.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.028+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f281002ae40 con 0x7f28080051c0 2024-04-26T20:22:02.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.028+0000 7f28177fe640 1 -- 172.21.15.37:0/3853500749 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810003bb0 con 0x7f28080051c0 2024-04-26T20:22:02.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.029+0000 7f281efaf640 1 -- 172.21.15.37:0/3853500749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28080051c0 msgr2=0x7f280800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.029+0000 7f281efaf640 1 --2- 172.21.15.37:0/3853500749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28080051c0 0x7f280800b9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2810001e20 tx=0x7f2808005830 comp rx=0 tx=0).stop 2024-04-26T20:22:02.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.029+0000 7f281efaf640 1 -- 172.21.15.37:0/3853500749 shutdown_connections 2024-04-26T20:22:02.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.029+0000 7f281efaf640 1 --2- 172.21.15.37:0/3853500749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28080051c0 0x7f280800b9c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.029+0000 7f281efaf640 1 -- 172.21.15.37:0/3853500749 >> 172.21.15.37:0/3853500749 conn(0x7f281809e510 msgr2=0x7f28180d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:02.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.029+0000 7f281efaf640 1 -- 172.21.15.37:0/3853500749 shutdown_connections 2024-04-26T20:22:02.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.029+0000 7f281efaf640 1 -- 172.21.15.37:0/3853500749 wait complete. 2024-04-26T20:22:02.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.030+0000 7f281efaf640 1 Processor -- start 2024-04-26T20:22:02.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.030+0000 7f281efaf640 1 -- start start 2024-04-26T20:22:02.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281efaf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28080051c0 0x7f2818157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281efaf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28181533c0 0x7f2818149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281efaf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28180cdb70 0x7f2818161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281efaf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f28180cfd50 con 0x7f28180cdb70 2024-04-26T20:22:02.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281efaf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28180d0660 con 0x7f28080051c0 2024-04-26T20:22:02.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281efaf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f281815f7e0 con 0x7f28181533c0 2024-04-26T20:22:02.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281d54c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28180cdb70 0x7f2818161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281d54c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28180cdb70 0x7f2818161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35348/0 (socket says 172.21.15.37:35348) 2024-04-26T20:22:02.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281d54c640 1 -- 172.21.15.37:0/3635404271 learned_addr learned my addr 172.21.15.37:0/3635404271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f2817fff640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28181533c0 0x7f2818149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281cd4b640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28080051c0 0x7f2818157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281d54c640 1 -- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28181533c0 msgr2=0x7f2818149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281d54c640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28181533c0 0x7f2818149540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281d54c640 1 -- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28080051c0 msgr2=0x7f2818157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281d54c640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28080051c0 0x7f2818157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281d54c640 1 -- 172.21.15.37:0/3635404271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28180cdff0 con 0x7f28180cdb70 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f2817fff640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28181533c0 0x7f2818149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.031+0000 7f281cd4b640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28080051c0 0x7f2818157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.032+0000 7f281d54c640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28180cdb70 0x7f2818161490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2810001f40 tx=0x7f28100014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.032+0000 7f28157fa640 1 -- 172.21.15.37:0/3635404271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810007dd0 con 0x7f28180cdb70 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.032+0000 7f28157fa640 1 -- 172.21.15.37:0/3635404271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2810003e10 con 0x7f28180cdb70 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.032+0000 7f281efaf640 1 -- 172.21.15.37:0/3635404271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2818153b80 con 0x7f28180cdb70 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.032+0000 7f28157fa640 1 -- 172.21.15.37:0/3635404271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810006b50 con 0x7f28180cdb70 2024-04-26T20:22:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.032+0000 7f281efaf640 1 -- 172.21.15.37:0/3635404271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28180ce270 con 0x7f28180cdb70 2024-04-26T20:22:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.033+0000 7f281efaf640 1 -- 172.21.15.37:0/3635404271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28180693e0 con 0x7f28180cdb70 2024-04-26T20:22:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.033+0000 7f28157fa640 1 -- 172.21.15.37:0/3635404271 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2810032440 con 0x7f28180cdb70 2024-04-26T20:22:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.034+0000 7f28157fa640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f27fc06c920 0x7f27fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.034+0000 7f28157fa640 1 -- 172.21.15.37:0/3635404271 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f28100aad40 con 0x7f28180cdb70 2024-04-26T20:22:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.034+0000 7f281cd4b640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f27fc06c920 0x7f27fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.034+0000 7f281cd4b640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f27fc06c920 0x7f27fc06ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f280c009a40 tx=0x7f280c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.038+0000 7f28157fa640 1 -- 172.21.15.37:0/3635404271 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28180cdff0 con 0x7f28180cdb70 2024-04-26T20:22:02.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.229+0000 7f281efaf640 1 -- 172.21.15.37:0/3635404271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f281806d1a0 con 0x7f28180cdb70 2024-04-26T20:22:02.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.230+0000 7f28157fa640 1 -- 172.21.15.37:0/3635404271 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f2810078b20 con 0x7f28180cdb70 2024-04-26T20:22:02.230 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:02.231 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:02.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 -- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f27fc06c920 msgr2=0x7f27fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f27fc06c920 0x7f27fc06ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f280c009a40 tx=0x7f280c008040 comp rx=0 tx=0).stop 2024-04-26T20:22:02.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 -- 172.21.15.37:0/3635404271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28180cdb70 msgr2=0x7f2818161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28180cdb70 0x7f2818161490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2810001f40 tx=0x7f28100014a0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 -- 172.21.15.37:0/3635404271 shutdown_connections 2024-04-26T20:22:02.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28180cdb70 0x7f2818161490 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f27fc06c920 0x7f27fc06ede0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28181533c0 0x7f2818149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 --2- 172.21.15.37:0/3635404271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28080051c0 0x7f2818157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 -- 172.21.15.37:0/3635404271 >> 172.21.15.37:0/3635404271 conn(0x7f281809e510 msgr2=0x7f28180b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:02.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 -- 172.21.15.37:0/3635404271 shutdown_connections 2024-04-26T20:22:02.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.232+0000 7f281efaf640 1 -- 172.21.15.37:0/3635404271 wait complete. 2024-04-26T20:22:02.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:02.244 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:02.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.347+0000 7fb0b395f640 1 Processor -- start 2024-04-26T20:22:02.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.347+0000 7fb0b395f640 1 -- start start 2024-04-26T20:22:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b395f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0ac15f470 0x7fb0ac0d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b395f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb0ac0d0d30 0x7fb0ac0d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b395f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0ac0d2ed0 con 0x7fb0ac0d1fe0 2024-04-26T20:22:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b395f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb0ac0d1190 con 0x7fb0ac0d0d30 2024-04-26T20:22:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b395f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ac0d31b0 con 0x7fb0ac15f470 2024-04-26T20:22:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b0efa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb0ac0d0d30 0x7fb0ac0d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b0efa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb0ac0d0d30 0x7fb0ac0d1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35350/0 (socket says 172.21.15.37:35350) 2024-04-26T20:22:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b0efa640 1 -- 172.21.15.37:0/1332028294 learned_addr learned my addr 172.21.15.37:0/1332028294 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b16fb640 1 --2- 172.21.15.37:0/1332028294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0ac15f470 0x7fb0ac0d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b0efa640 1 -- 172.21.15.37:0/1332028294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0ac15f470 msgr2=0x7fb0ac0d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b0efa640 1 --2- 172.21.15.37:0/1332028294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0ac15f470 0x7fb0ac0d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b0efa640 1 -- 172.21.15.37:0/1332028294 >> v1:172.21.15.134:6789/0 conn(0x7fb0ac0d1fe0 legacy=0x7fb0ac162fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:02.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.348+0000 7fb0b0efa640 1 -- 172.21.15.37:0/1332028294 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac0d3490 con 0x7fb0ac0d0d30 2024-04-26T20:22:02.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.349+0000 7fb0b0efa640 1 --2- 172.21.15.37:0/1332028294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac0d0d30 0x7fb0ac0d1aa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0098a0 tx=0x7fb09c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2761cf7472d768 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.349+0000 7fb0a3fff640 1 -- 172.21.15.37:0/1332028294 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c037070 con 0x7fb0ac0d0d30 2024-04-26T20:22:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.349+0000 7fb0a3fff640 1 -- 172.21.15.37:0/1332028294 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb09c032440 con 0x7fb0ac0d0d30 2024-04-26T20:22:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.349+0000 7fb0a3fff640 1 -- 172.21.15.37:0/1332028294 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c032810 con 0x7fb0ac0d0d30 2024-04-26T20:22:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.350+0000 7fb0b395f640 1 -- 172.21.15.37:0/1332028294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac0d0d30 msgr2=0x7fb0ac0d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.350+0000 7fb0b395f640 1 --2- 172.21.15.37:0/1332028294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac0d0d30 0x7fb0ac0d1aa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0098a0 tx=0x7fb09c02a250 comp rx=0 tx=0).stop 2024-04-26T20:22:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.350+0000 7fb0b395f640 1 -- 172.21.15.37:0/1332028294 shutdown_connections 2024-04-26T20:22:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.350+0000 7fb0b395f640 1 --2- 172.21.15.37:0/1332028294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac0d0d30 0x7fb0ac0d1aa0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.350+0000 7fb0b395f640 1 --2- 172.21.15.37:0/1332028294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0ac15f470 0x7fb0ac0d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.350+0000 7fb0b395f640 1 -- 172.21.15.37:0/1332028294 >> 172.21.15.37:0/1332028294 conn(0x7fb0ac09e510 msgr2=0x7fb0ac0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:02.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.350+0000 7fb0b395f640 1 -- 172.21.15.37:0/1332028294 shutdown_connections 2024-04-26T20:22:02.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.350+0000 7fb0b395f640 1 -- 172.21.15.37:0/1332028294 wait complete. 2024-04-26T20:22:02.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.351+0000 7fb0b395f640 1 Processor -- start 2024-04-26T20:22:02.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.351+0000 7fb0b395f640 1 -- start start 2024-04-26T20:22:02.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b395f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0ac0d0d30 0x7fb0ac1f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b395f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0ac0d1fe0 0x7fb0ac1e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b395f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15f470 0x7fb0ac1ea360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b395f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac15f850 con 0x7fb0ac15f470 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b395f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac0d3a50 con 0x7fb0ac0d1fe0 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b395f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ac0d3f70 con 0x7fb0ac0d0d30 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b1efc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15f470 0x7fb0ac1ea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b1efc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15f470 0x7fb0ac1ea360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35356/0 (socket says 172.21.15.37:35356) 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b1efc640 1 -- 172.21.15.37:0/1465219364 learned_addr learned my addr 172.21.15.37:0/1465219364 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b16fb640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0ac0d0d30 0x7fb0ac1f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b0efa640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0ac0d1fe0 0x7fb0ac1e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b1efc640 1 -- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0ac0d0d30 msgr2=0x7fb0ac1f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b1efc640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0ac0d0d30 0x7fb0ac1f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b1efc640 1 -- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0ac0d1fe0 msgr2=0x7fb0ac1e9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b1efc640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0ac0d1fe0 0x7fb0ac1e9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b1efc640 1 -- 172.21.15.37:0/1465219364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac0cf2d0 con 0x7fb0ac15f470 2024-04-26T20:22:02.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.352+0000 7fb0b1efc640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15f470 0x7fb0ac1ea360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400b410 tx=0x7fb0a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.353+0000 7fb0a1ffb640 1 -- 172.21.15.37:0/1465219364 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4015030 con 0x7fb0ac15f470 2024-04-26T20:22:02.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.353+0000 7fb0a1ffb640 1 -- 172.21.15.37:0/1465219364 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a4004020 con 0x7fb0ac15f470 2024-04-26T20:22:02.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.353+0000 7fb0b395f640 1 -- 172.21.15.37:0/1465219364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac0c8fe0 con 0x7fb0ac15f470 2024-04-26T20:22:02.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.353+0000 7fb0a1ffb640 1 -- 172.21.15.37:0/1465219364 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4004410 con 0x7fb0ac15f470 2024-04-26T20:22:02.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.353+0000 7fb0b395f640 1 -- 172.21.15.37:0/1465219364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac0c95a0 con 0x7fb0ac15f470 2024-04-26T20:22:02.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.354+0000 7fb0b395f640 1 -- 172.21.15.37:0/1465219364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ac06a360 con 0x7fb0ac15f470 2024-04-26T20:22:02.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.354+0000 7fb0a1ffb640 1 -- 172.21.15.37:0/1465219364 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0a4018070 con 0x7fb0ac15f470 2024-04-26T20:22:02.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.355+0000 7fb0a1ffb640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb09006c8f0 0x7fb09006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.355+0000 7fb0a1ffb640 1 -- 172.21.15.37:0/1465219364 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb0a408b450 con 0x7fb0ac15f470 2024-04-26T20:22:02.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.355+0000 7fb0b16fb640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb09006c8f0 0x7fb09006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.355+0000 7fb0b16fb640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb09006c8f0 0x7fb09006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800f360 tx=0x7fb0a8007710 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.359+0000 7fb0a1ffb640 1 -- 172.21.15.37:0/1465219364 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0a40592c0 con 0x7fb0ac15f470 2024-04-26T20:22:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.546+0000 7fb0b395f640 1 -- 172.21.15.37:0/1465219364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0ac0d5870 con 0x7fb0ac15f470 2024-04-26T20:22:02.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.547+0000 7fb0a1ffb640 1 -- 172.21.15.37:0/1465219364 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb0a405d5c0 con 0x7fb0ac15f470 2024-04-26T20:22:02.547 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:02.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 -- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb09006c8f0 msgr2=0x7fb09006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb09006c8f0 0x7fb09006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800f360 tx=0x7fb0a8007710 comp rx=0 tx=0).stop 2024-04-26T20:22:02.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 -- 172.21.15.37:0/1465219364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15f470 msgr2=0x7fb0ac1ea360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15f470 0x7fb0ac1ea360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400b410 tx=0x7fb0a400b8d0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 -- 172.21.15.37:0/1465219364 shutdown_connections 2024-04-26T20:22:02.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15f470 0x7fb0ac1ea360 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0ac0d1fe0 0x7fb0ac1e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb09006c8f0 0x7fb09006edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 --2- 172.21.15.37:0/1465219364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0ac0d0d30 0x7fb0ac1f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 -- 172.21.15.37:0/1465219364 >> 172.21.15.37:0/1465219364 conn(0x7fb0ac09e510 msgr2=0x7fb0ac15dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:02.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 -- 172.21.15.37:0/1465219364 shutdown_connections 2024-04-26T20:22:02.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.549+0000 7fb0b395f640 1 -- 172.21.15.37:0/1465219364 wait complete. 2024-04-26T20:22:02.560 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:02.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.663+0000 7f6c144c4640 1 Processor -- start 2024-04-26T20:22:02.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.663+0000 7f6c144c4640 1 -- start start 2024-04-26T20:22:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c144c4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c0c15f470 0x7f6c0c0d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c144c4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c0c0d0d30 0x7f6c0c0d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c144c4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6c0c0d1fe0 0x7f6c0c162fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c144c4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6c0c0d1190 con 0x7f6c0c15f470 2024-04-26T20:22:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c144c4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6c0c0d2ed0 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c144c4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0c0d31b0 con 0x7f6c0c0d0d30 2024-04-26T20:22:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c12260640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c0c15f470 0x7f6c0c0d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c12260640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c0c15f470 0x7f6c0c0d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35364/0 (socket says 172.21.15.37:35364) 2024-04-26T20:22:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c12260640 1 -- 172.21.15.37:0/852094333 learned_addr learned my addr 172.21.15.37:0/852094333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c12a61640 1 --2- 172.21.15.37:0/852094333 >> v2:172.21.15.134:3300/0 conn(0x7f6c0c0d1fe0 0x7f6c0c162fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c11a5f640 1 --2- 172.21.15.37:0/852094333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c0c0d0d30 0x7f6c0c0d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c12260640 1 -- 172.21.15.37:0/852094333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c0c0d0d30 msgr2=0x7f6c0c0d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c12260640 1 --2- 172.21.15.37:0/852094333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c0c0d0d30 0x7f6c0c0d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c12260640 1 -- 172.21.15.37:0/852094333 >> v2:172.21.15.134:3300/0 conn(0x7f6c0c0d1fe0 msgr2=0x7f6c0c162fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c12260640 1 --2- 172.21.15.37:0/852094333 >> v2:172.21.15.134:3300/0 conn(0x7f6c0c0d1fe0 0x7f6c0c162fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.664+0000 7f6c12260640 1 -- 172.21.15.37:0/852094333 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0c0d3490 con 0x7f6c0c15f470 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.665+0000 7f6c12260640 1 --2- 172.21.15.37:0/852094333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c0c15f470 0x7f6c0c0d07f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009870 tx=0x7f6c0802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436d4df1ec7c6811 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.665+0000 7f6c1125e640 1 -- 172.21.15.37:0/852094333 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08037070 con 0x7f6c0c15f470 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.665+0000 7f6c1125e640 1 -- 172.21.15.37:0/852094333 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c08032440 con 0x7f6c0c15f470 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.665+0000 7f6c1125e640 1 -- 172.21.15.37:0/852094333 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08032810 con 0x7f6c0c15f470 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.665+0000 7f6c144c4640 1 -- 172.21.15.37:0/852094333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c0c15f470 msgr2=0x7f6c0c0d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.665+0000 7f6c144c4640 1 --2- 172.21.15.37:0/852094333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c0c15f470 0x7f6c0c0d07f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009870 tx=0x7f6c0802a210 comp rx=0 tx=0).stop 2024-04-26T20:22:02.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.666+0000 7f6c144c4640 1 -- 172.21.15.37:0/852094333 shutdown_connections 2024-04-26T20:22:02.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.666+0000 7f6c144c4640 1 --2- 172.21.15.37:0/852094333 >> v2:172.21.15.134:3300/0 conn(0x7f6c0c0d1fe0 0x7f6c0c162fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.666+0000 7f6c144c4640 1 --2- 172.21.15.37:0/852094333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c0c0d0d30 0x7f6c0c0d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.666+0000 7f6c144c4640 1 --2- 172.21.15.37:0/852094333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c0c15f470 0x7f6c0c0d07f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.666+0000 7f6c144c4640 1 -- 172.21.15.37:0/852094333 >> 172.21.15.37:0/852094333 conn(0x7f6c0c09e510 msgr2=0x7f6c0c0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:02.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.666+0000 7f6c144c4640 1 -- 172.21.15.37:0/852094333 shutdown_connections 2024-04-26T20:22:02.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.666+0000 7f6c144c4640 1 -- 172.21.15.37:0/852094333 wait complete. 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.667+0000 7f6c144c4640 1 Processor -- start 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.667+0000 7f6c144c4640 1 -- start start 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.667+0000 7f6c144c4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c0c0d0d30 0x7f6c0c15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.667+0000 7f6c144c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c0c0d1fe0 0x7f6c0c1601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.667+0000 7f6c144c4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c0c15f470 0x7f6c0c0ba840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.667+0000 7f6c144c4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c15f850 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.667+0000 7f6c144c4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c0d3a50 con 0x7f6c0c15f470 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.667+0000 7f6c144c4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0c0d3f70 con 0x7f6c0c0d0d30 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c11a5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c0c0d1fe0 0x7f6c0c1601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c11a5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c0c0d1fe0 0x7f6c0c1601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35374/0 (socket says 172.21.15.37:35374) 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c11a5f640 1 -- 172.21.15.37:0/963259007 learned_addr learned my addr 172.21.15.37:0/963259007 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c12260640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c0c0d0d30 0x7f6c0c15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c12a61640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c0c15f470 0x7f6c0c0ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c11a5f640 1 -- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c0c0d0d30 msgr2=0x7f6c0c15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c11a5f640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c0c0d0d30 0x7f6c0c15bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c11a5f640 1 -- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c0c15f470 msgr2=0x7f6c0c0ba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c11a5f640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c0c15f470 0x7f6c0c0ba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c11a5f640 1 -- 172.21.15.37:0/963259007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0c160710 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c11a5f640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c0c0d1fe0 0x7f6c0c1601d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00d5f0 tx=0x7f6bfc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c02ffd640 1 -- 172.21.15.37:0/963259007 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0125f0 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c02ffd640 1 -- 172.21.15.37:0/963259007 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bfc019070 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c144c4640 1 -- 172.21.15.37:0/963259007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0c0cd7c0 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c02ffd640 1 -- 172.21.15.37:0/963259007 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0129d0 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.668+0000 7f6c144c4640 1 -- 172.21.15.37:0/963259007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c0c0cdd80 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.669+0000 7f6c144c4640 1 -- 172.21.15.37:0/963259007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c0c069af0 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.670+0000 7f6c02ffd640 1 -- 172.21.15.37:0/963259007 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6bfc012b70 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.670+0000 7f6c02ffd640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6bf006c920 0x7f6bf006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.670+0000 7f6c02ffd640 1 -- 172.21.15.37:0/963259007 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6bfc08bf00 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.670+0000 7f6c12260640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6bf006c920 0x7f6bf006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.671+0000 7f6c12260640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6bf006c920 0x7f6bf006ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6c08034000 tx=0x7f6c0802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.674+0000 7f6c02ffd640 1 -- 172.21.15.37:0/963259007 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6c0c069af0 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.857+0000 7f6c144c4640 1 -- 172.21.15.37:0/963259007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6c0c06a360 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.857+0000 7f6c02ffd640 1 -- 172.21.15.37:0/963259007 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f6bfc05a330 con 0x7f6c0c0d1fe0 2024-04-26T20:22:02.858 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:02.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 -- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6bf006c920 msgr2=0x7f6bf006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6bf006c920 0x7f6bf006ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6c08034000 tx=0x7f6c0802e000 comp rx=0 tx=0).stop 2024-04-26T20:22:02.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 -- 172.21.15.37:0/963259007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c0c0d1fe0 msgr2=0x7f6c0c1601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c0c0d1fe0 0x7f6c0c1601d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00d5f0 tx=0x7f6bfc00dab0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 -- 172.21.15.37:0/963259007 shutdown_connections 2024-04-26T20:22:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c0c15f470 0x7f6c0c0ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c0c0d1fe0 0x7f6c0c1601d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6bf006c920 0x7f6bf006ede0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 --2- 172.21.15.37:0/963259007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c0c0d0d30 0x7f6c0c15bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 -- 172.21.15.37:0/963259007 >> 172.21.15.37:0/963259007 conn(0x7f6c0c09e510 msgr2=0x7f6c0c0cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 -- 172.21.15.37:0/963259007 shutdown_connections 2024-04-26T20:22:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.859+0000 7f6c144c4640 1 -- 172.21.15.37:0/963259007 wait complete. 2024-04-26T20:22:02.870 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.974+0000 7f4a943af640 1 Processor -- start 2024-04-26T20:22:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.974+0000 7f4a943af640 1 -- start start 2024-04-26T20:22:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.975+0000 7f4a943af640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4a8c0d5a80 0x7f4a8c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.975+0000 7f4a943af640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a8c0b2760 con 0x7f4a8c15f400 2024-04-26T20:22:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.975+0000 7f4a943af640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a8c0b2a00 con 0x7f4a8c0b4b20 2024-04-26T20:22:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.975+0000 7f4a943af640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8c0b43e0 con 0x7f4a8c0d5a80 2024-04-26T20:22:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.975+0000 7f4a9194a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4a8c0d5a80 0x7f4a8c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.975+0000 7f4a9214b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4a8c15f400 0x7f4a8c0b37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44416/0 (socket says 172.21.15.37:44416) 2024-04-26T20:22:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.975+0000 7f4a9214b640 1 -- 172.21.15.37:0/3355692792 learned_addr learned my addr 172.21.15.37:0/3355692792 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:02.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.975+0000 7f4a9194a640 1 -- 172.21.15.37:0/3355692792 >> v1:172.21.15.134:6789/0 conn(0x7f4a8c0b4b20 legacy=0x7f4a8c1630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:02.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.975+0000 7f4a9194a640 1 -- 172.21.15.37:0/3355692792 >> v1:172.21.15.37:6789/0 conn(0x7f4a8c15f400 legacy=0x7f4a8c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.975+0000 7f4a9194a640 1 -- 172.21.15.37:0/3355692792 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c0b2ce0 con 0x7f4a8c0d5a80 2024-04-26T20:22:02.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.975+0000 7f4a91149640 1 -- 172.21.15.37:0/3355692792 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4121774123 0 0) 0x7f4a8c0b2760 con 0x7f4a8c15f400 2024-04-26T20:22:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.976+0000 7f4a9194a640 1 --2- 172.21.15.37:0/3355692792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a8c0d5a80 0x7f4a8c0b45e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0097a0 tx=0x7f4a7c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27abc5735db1983b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.976+0000 7f4a91149640 1 -- 172.21.15.37:0/3355692792 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c037070 con 0x7f4a8c0d5a80 2024-04-26T20:22:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.976+0000 7f4a91149640 1 -- 172.21.15.37:0/3355692792 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7c02ae50 con 0x7f4a8c0d5a80 2024-04-26T20:22:02.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.976+0000 7f4a91149640 1 -- 172.21.15.37:0/3355692792 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c032660 con 0x7f4a8c0d5a80 2024-04-26T20:22:02.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.976+0000 7f4a943af640 1 -- 172.21.15.37:0/3355692792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a8c0d5a80 msgr2=0x7f4a8c0b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.976+0000 7f4a943af640 1 --2- 172.21.15.37:0/3355692792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a8c0d5a80 0x7f4a8c0b45e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0097a0 tx=0x7f4a7c02a140 comp rx=0 tx=0).stop 2024-04-26T20:22:02.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.976+0000 7f4a943af640 1 -- 172.21.15.37:0/3355692792 shutdown_connections 2024-04-26T20:22:02.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.976+0000 7f4a943af640 1 --2- 172.21.15.37:0/3355692792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a8c0d5a80 0x7f4a8c0b45e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.977+0000 7f4a943af640 1 -- 172.21.15.37:0/3355692792 >> 172.21.15.37:0/3355692792 conn(0x7f4a8c09e4f0 msgr2=0x7f4a8c0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:02.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.977+0000 7f4a943af640 1 -- 172.21.15.37:0/3355692792 shutdown_connections 2024-04-26T20:22:02.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.977+0000 7f4a943af640 1 -- 172.21.15.37:0/3355692792 wait complete. 2024-04-26T20:22:02.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.978+0000 7f4a943af640 1 Processor -- start 2024-04-26T20:22:02.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.978+0000 7f4a943af640 1 -- start start 2024-04-26T20:22:02.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.978+0000 7f4a943af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a8c0b4b20 0x7f4a8c154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.978+0000 7f4a943af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a8c0d5a80 0x7f4a8c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.978+0000 7f4a943af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a8c15f400 0x7f4a8c1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.978+0000 7f4a943af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c0b32a0 con 0x7f4a8c0b4b20 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.978+0000 7f4a943af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c0b3ed0 con 0x7f4a8c0d5a80 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.978+0000 7f4a943af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c0b4f30 con 0x7f4a8c15f400 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9214b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a8c0b4b20 0x7f4a8c154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9214b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a8c0b4b20 0x7f4a8c154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35392/0 (socket says 172.21.15.37:35392) 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9214b640 1 -- 172.21.15.37:0/3582858130 learned_addr learned my addr 172.21.15.37:0/3582858130 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9194a640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a8c0d5a80 0x7f4a8c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9294c640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a8c15f400 0x7f4a8c1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9214b640 1 -- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a8c15f400 msgr2=0x7f4a8c1511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9214b640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a8c15f400 0x7f4a8c1511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9214b640 1 -- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a8c0d5a80 msgr2=0x7f4a8c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9214b640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a8c0d5a80 0x7f4a8c0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9214b640 1 -- 172.21.15.37:0/3582858130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c154fe0 con 0x7f4a8c0b4b20 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9194a640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a8c0d5a80 0x7f4a8c0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:02.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9294c640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a8c15f400 0x7f4a8c1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a9214b640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a8c0b4b20 0x7f4a8c154aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c4a0 tx=0x7f4a8800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a82ffd640 1 -- 172.21.15.37:0/3582858130 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88017070 con 0x7f4a8c0b4b20 2024-04-26T20:22:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a82ffd640 1 -- 172.21.15.37:0/3582858130 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a88005ce0 con 0x7f4a8c0b4b20 2024-04-26T20:22:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a82ffd640 1 -- 172.21.15.37:0/3582858130 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880072b0 con 0x7f4a8c0b4b20 2024-04-26T20:22:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a943af640 1 -- 172.21.15.37:0/3582858130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8c0ba200 con 0x7f4a8c0b4b20 2024-04-26T20:22:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.979+0000 7f4a943af640 1 -- 172.21.15.37:0/3582858130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8c0ba7c0 con 0x7f4a8c0b4b20 2024-04-26T20:22:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.980+0000 7f4a943af640 1 -- 172.21.15.37:0/3582858130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a8c06a360 con 0x7f4a8c0b4b20 2024-04-26T20:22:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.981+0000 7f4a82ffd640 1 -- 172.21.15.37:0/3582858130 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a8801a070 con 0x7f4a8c0b4b20 2024-04-26T20:22:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.981+0000 7f4a82ffd640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4a7006c8f0 0x7f4a7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.981+0000 7f4a82ffd640 1 -- 172.21.15.37:0/3582858130 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a8808af10 con 0x7f4a8c0b4b20 2024-04-26T20:22:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.981+0000 7f4a9194a640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4a7006c8f0 0x7f4a7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.982+0000 7f4a9194a640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4a7006c8f0 0x7f4a7006edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c034000 tx=0x7f4a7c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:02.985+0000 7f4a82ffd640 1 -- 172.21.15.37:0/3582858130 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4a88059340 con 0x7f4a8c0b4b20 2024-04-26T20:22:03.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.169+0000 7f4a943af640 1 -- 172.21.15.37:0/3582858130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4a8c0d28f0 con 0x7f4a8c0b4b20 2024-04-26T20:22:03.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.170+0000 7f4a82ffd640 1 -- 172.21.15.37:0/3582858130 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f4a88058c40 con 0x7f4a8c0b4b20 2024-04-26T20:22:03.170 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 -- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4a7006c8f0 msgr2=0x7f4a7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4a7006c8f0 0x7f4a7006edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c034000 tx=0x7f4a7c02e000 comp rx=0 tx=0).stop 2024-04-26T20:22:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 -- 172.21.15.37:0/3582858130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a8c0b4b20 msgr2=0x7f4a8c154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a8c0b4b20 0x7f4a8c154aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c4a0 tx=0x7f4a8800c960 comp rx=0 tx=0).stop 2024-04-26T20:22:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 -- 172.21.15.37:0/3582858130 shutdown_connections 2024-04-26T20:22:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4a7006c8f0 0x7f4a7006edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a8c15f400 0x7f4a8c1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a8c0d5a80 0x7f4a8c0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 --2- 172.21.15.37:0/3582858130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a8c0b4b20 0x7f4a8c154aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 -- 172.21.15.37:0/3582858130 >> 172.21.15.37:0/3582858130 conn(0x7f4a8c09e4f0 msgr2=0x7f4a8c0d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 -- 172.21.15.37:0/3582858130 shutdown_connections 2024-04-26T20:22:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.172+0000 7f4a943af640 1 -- 172.21.15.37:0/3582858130 wait complete. 2024-04-26T20:22:03.183 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:22:03.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.282+0000 7f9a00d14640 1 Processor -- start 2024-04-26T20:22:03.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.282+0000 7f9a00d14640 1 -- start start 2024-04-26T20:22:03.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.283+0000 7f9a00d14640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc15f380 0x7f99fc0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.283+0000 7f9a00d14640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f99fc15f760 0x7f99fc0d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.283+0000 7f9a00d14640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99fc0d3de0 con 0x7f99fc0d5b50 2024-04-26T20:22:03.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.283+0000 7f9a00d14640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f99fc0d43a0 con 0x7f99fc15f760 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.283+0000 7f9a00d14640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f99fc0d40c0 con 0x7f99fc15f380 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.283+0000 7f99f9d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f99fc15f760 0x7f99fc0d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.283+0000 7f99f9d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f99fc15f760 0x7f99fc0d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35398/0 (socket says 172.21.15.37:35398) 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.283+0000 7f99f9d9b640 1 -- 172.21.15.37:0/3108495147 learned_addr learned my addr 172.21.15.37:0/3108495147 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.283+0000 7f99fa59c640 1 --2- 172.21.15.37:0/3108495147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc15f380 0x7f99fc0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.284+0000 7f99f9d9b640 1 -- 172.21.15.37:0/3108495147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc15f380 msgr2=0x7f99fc0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.284+0000 7f99f9d9b640 1 --2- 172.21.15.37:0/3108495147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc15f380 0x7f99fc0d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.284+0000 7f99f9d9b640 1 -- 172.21.15.37:0/3108495147 >> v1:172.21.15.134:6789/0 conn(0x7f99fc0d5b50 legacy=0x7f99fc0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.284+0000 7f99f9d9b640 1 -- 172.21.15.37:0/3108495147 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc0b8280 con 0x7f99fc15f760 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.284+0000 7f99fa59c640 1 --2- 172.21.15.37:0/3108495147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc15f380 0x7f99fc0d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.284+0000 7f99f9d9b640 1 --2- 172.21.15.37:0/3108495147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99fc15f760 0x7f99fc0d2110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009870 tx=0x7f99e402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313ed66a63e33019 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.284+0000 7f99f959a640 1 -- 172.21.15.37:0/3108495147 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4037070 con 0x7f99fc15f760 2024-04-26T20:22:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.284+0000 7f99f959a640 1 -- 172.21.15.37:0/3108495147 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e402ae50 con 0x7f99fc15f760 2024-04-26T20:22:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.284+0000 7f99f959a640 1 -- 172.21.15.37:0/3108495147 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4032860 con 0x7f99fc15f760 2024-04-26T20:22:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.285+0000 7f9a00d14640 1 -- 172.21.15.37:0/3108495147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99fc15f760 msgr2=0x7f99fc0d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.285+0000 7f9a00d14640 1 --2- 172.21.15.37:0/3108495147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99fc15f760 0x7f99fc0d2110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009870 tx=0x7f99e402a300 comp rx=0 tx=0).stop 2024-04-26T20:22:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.285+0000 7f9a00d14640 1 -- 172.21.15.37:0/3108495147 shutdown_connections 2024-04-26T20:22:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.285+0000 7f9a00d14640 1 --2- 172.21.15.37:0/3108495147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99fc15f760 0x7f99fc0d2110 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.285+0000 7f9a00d14640 1 --2- 172.21.15.37:0/3108495147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc15f380 0x7f99fc0d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.285+0000 7f9a00d14640 1 -- 172.21.15.37:0/3108495147 >> 172.21.15.37:0/3108495147 conn(0x7f99fc09e560 msgr2=0x7f99fc0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.285+0000 7f9a00d14640 1 -- 172.21.15.37:0/3108495147 shutdown_connections 2024-04-26T20:22:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.285+0000 7f9a00d14640 1 -- 172.21.15.37:0/3108495147 wait complete. 2024-04-26T20:22:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.286+0000 7f9a00d14640 1 Processor -- start 2024-04-26T20:22:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.286+0000 7f9a00d14640 1 -- start start 2024-04-26T20:22:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f9a00d14640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0d5b50 0x7f99fc1622c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f9a00d14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc15f380 0x7f99fc1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f9a00d14640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99fc15f760 0x7f99fc153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f9a00d14640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc0d1280 con 0x7f99fc15f760 2024-04-26T20:22:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f9a00d14640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc0d1100 con 0x7f99fc15f380 2024-04-26T20:22:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f9a00d14640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f99fc0b9a90 con 0x7f99fc0d5b50 2024-04-26T20:22:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f99fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99fc15f760 0x7f99fc153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f99fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99fc15f760 0x7f99fc153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35406/0 (socket says 172.21.15.37:35406) 2024-04-26T20:22:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f99fad9d640 1 -- 172.21.15.37:0/1672833137 learned_addr learned my addr 172.21.15.37:0/1672833137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:03.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f99f9d9b640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc15f380 0x7f99fc1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f99fa59c640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0d5b50 0x7f99fc1622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.287+0000 7f99fad9d640 1 -- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0d5b50 msgr2=0x7f99fc1622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.288+0000 7f99fad9d640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0d5b50 0x7f99fc1622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.288+0000 7f99fad9d640 1 -- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc15f380 msgr2=0x7f99fc1532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.288+0000 7f99fad9d640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc15f380 0x7f99fc1532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.288+0000 7f99fad9d640 1 -- 172.21.15.37:0/1672833137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc0cc400 con 0x7f99fc15f760 2024-04-26T20:22:03.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.288+0000 7f99fa59c640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0d5b50 0x7f99fc1622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:03.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.288+0000 7f99fad9d640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99fc15f760 0x7f99fc153820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c710 tx=0x7f99ec00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:03.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.288+0000 7f99f2ffd640 1 -- 172.21.15.37:0/1672833137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec016020 con 0x7f99fc15f760 2024-04-26T20:22:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.288+0000 7f99f2ffd640 1 -- 172.21.15.37:0/1672833137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ec006d50 con 0x7f99fc15f760 2024-04-26T20:22:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.288+0000 7f9a00d14640 1 -- 172.21.15.37:0/1672833137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99fc0cc6e0 con 0x7f99fc15f760 2024-04-26T20:22:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.288+0000 7f99f2ffd640 1 -- 172.21.15.37:0/1672833137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec003bb0 con 0x7f99fc15f760 2024-04-26T20:22:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.288+0000 7f9a00d14640 1 -- 172.21.15.37:0/1672833137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99fc0ccca0 con 0x7f99fc15f760 2024-04-26T20:22:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.289+0000 7f9a00d14640 1 -- 172.21.15.37:0/1672833137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99fc06a360 con 0x7f99fc15f760 2024-04-26T20:22:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.289+0000 7f99f2ffd640 1 -- 172.21.15.37:0/1672833137 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99ec00a850 con 0x7f99fc15f760 2024-04-26T20:22:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.290+0000 7f99f2ffd640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f99dc06c8f0 0x7f99dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.290+0000 7f99f2ffd640 1 -- 172.21.15.37:0/1672833137 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f99ec08b6c0 con 0x7f99fc15f760 2024-04-26T20:22:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.290+0000 7f99fa59c640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f99dc06c8f0 0x7f99dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.290+0000 7f99fa59c640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f99dc06c8f0 0x7f99dc06edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f99f400aa30 tx=0x7f99f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:03.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.294+0000 7f99f2ffd640 1 -- 172.21.15.37:0/1672833137 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f99ec020020 con 0x7f99fc15f760 2024-04-26T20:22:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.518+0000 7f9a00d14640 1 -- 172.21.15.37:0/1672833137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99fc0b7190 con 0x7f99fc15f760 2024-04-26T20:22:03.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.518+0000 7f99f2ffd640 1 -- 172.21.15.37:0/1672833137 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7f99ec059b80 con 0x7f99fc15f760 2024-04-26T20:22:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:03.519 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2911741012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2911741012},{"type":"v1","addr":"172.21.15.134:6836","nonce":2911741012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1600483467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1600483467},{"type":"v1","addr":"172.21.15.37:6835","nonce":1600483467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2722614558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2722614558},{"type":"v1","addr":"172.21.15.134:6837","nonce":2722614558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:21:49.237855+0000","modified":"2024-04-26T20:21:52.255148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3920647125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3920647125},{"type":"v1","addr":"172.21.15.37:6834","nonce":3920647125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-04-26T20:22:03.519 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-26T20:22:03.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 -- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f99dc06c8f0 msgr2=0x7f99dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f99dc06c8f0 0x7f99dc06edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f99f400aa30 tx=0x7f99f4008040 comp rx=0 tx=0).stop 2024-04-26T20:22:03.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 -- 172.21.15.37:0/1672833137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99fc15f760 msgr2=0x7f99fc153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99fc15f760 0x7f99fc153820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c710 tx=0x7f99ec00cbd0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 -- 172.21.15.37:0/1672833137 shutdown_connections 2024-04-26T20:22:03.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99fc15f760 0x7f99fc153820 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc15f380 0x7f99fc1532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f99dc06c8f0 0x7f99dc06edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 --2- 172.21.15.37:0/1672833137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0d5b50 0x7f99fc1622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 -- 172.21.15.37:0/1672833137 >> 172.21.15.37:0/1672833137 conn(0x7f99fc09e560 msgr2=0x7f99fc06ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:03.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 -- 172.21.15.37:0/1672833137 shutdown_connections 2024-04-26T20:22:03.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.520+0000 7f9a00d14640 1 -- 172.21.15.37:0/1672833137 wait complete. 2024-04-26T20:22:03.531 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:03.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.631+0000 7f267b3af640 1 Processor -- start 2024-04-26T20:22:03.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.631+0000 7f267b3af640 1 -- start start 2024-04-26T20:22:03.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.632+0000 7f267b3af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f267406e360 0x7f26740cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.632+0000 7f267b3af640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26740d11d0 con 0x7f26740d0060 2024-04-26T20:22:03.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.632+0000 7f267b3af640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26740d1470 con 0x7f267415f2a0 2024-04-26T20:22:03.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.632+0000 7f267b3af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f267409c870 con 0x7f267406e360 2024-04-26T20:22:03.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.632+0000 7f267894a640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f26740d0060 0x7f26740cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44418/0 (socket says 172.21.15.37:44418) 2024-04-26T20:22:03.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.632+0000 7f267894a640 1 -- 172.21.15.37:0/3422546987 learned_addr learned my addr 172.21.15.37:0/3422546987 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:03.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.633+0000 7f267994c640 1 --2- 172.21.15.37:0/3422546987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f267406e360 0x7f26740cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.633+0000 7f2673fff640 1 -- 172.21.15.37:0/3422546987 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692492263 0 0) 0x7f26740d11d0 con 0x7f26740d0060 2024-04-26T20:22:03.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.633+0000 7f2673fff640 1 -- 172.21.15.37:0/3422546987 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f265c003520 con 0x7f26740d0060 2024-04-26T20:22:03.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.633+0000 7f2673fff640 1 -- 172.21.15.37:0/3422546987 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3162157460 0 0) 0x7f26740d1470 con 0x7f267415f2a0 2024-04-26T20:22:03.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.633+0000 7f2673fff640 1 -- 172.21.15.37:0/3422546987 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26740d11d0 con 0x7f267415f2a0 2024-04-26T20:22:03.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.633+0000 7f267994c640 1 -- 172.21.15.37:0/3422546987 >> v1:172.21.15.134:6789/0 conn(0x7f267415f2a0 legacy=0x7f26740cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.633+0000 7f267994c640 1 -- 172.21.15.37:0/3422546987 >> v1:172.21.15.37:6789/0 conn(0x7f26740d0060 legacy=0x7f26740cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.633+0000 7f267994c640 1 -- 172.21.15.37:0/3422546987 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26740d1750 con 0x7f267406e360 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.634+0000 7f267994c640 1 --2- 172.21.15.37:0/3422546987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f267406e360 0x7f26740cdcf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2664009dc0 tx=0x7f266402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43b724becabf669f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.634+0000 7f2673fff640 1 -- 172.21.15.37:0/3422546987 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664007740 con 0x7f267406e360 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.634+0000 7f2673fff640 1 -- 172.21.15.37:0/3422546987 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26640078e0 con 0x7f267406e360 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.634+0000 7f2673fff640 1 -- 172.21.15.37:0/3422546987 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664007c90 con 0x7f267406e360 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.634+0000 7f267b3af640 1 -- 172.21.15.37:0/3422546987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f267406e360 msgr2=0x7f26740cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.634+0000 7f267b3af640 1 --2- 172.21.15.37:0/3422546987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f267406e360 0x7f26740cdcf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2664009dc0 tx=0x7f266402a6d0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.634+0000 7f267b3af640 1 -- 172.21.15.37:0/3422546987 shutdown_connections 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.634+0000 7f267b3af640 1 --2- 172.21.15.37:0/3422546987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f267406e360 0x7f26740cdcf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.634+0000 7f267b3af640 1 -- 172.21.15.37:0/3422546987 >> 172.21.15.37:0/3422546987 conn(0x7f267409ba50 msgr2=0x7f267409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.634+0000 7f267b3af640 1 -- 172.21.15.37:0/3422546987 shutdown_connections 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.635+0000 7f267b3af640 1 -- 172.21.15.37:0/3422546987 wait complete. 2024-04-26T20:22:03.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.636+0000 7f267b3af640 1 Processor -- start 2024-04-26T20:22:03.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.636+0000 7f267b3af640 1 -- start start 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.636+0000 7f267b3af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f267406e360 0x7f2674159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.636+0000 7f267b3af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26740d0060 0x7f26741e9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.636+0000 7f267b3af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f267415f2a0 0x7f26741ea120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.636+0000 7f267b3af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26740d1ff0 con 0x7f267415f2a0 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.636+0000 7f267b3af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26740d1d10 con 0x7f267406e360 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.636+0000 7f267b3af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f26740d2510 con 0x7f26740d0060 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267994c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f267415f2a0 0x7f26741ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267994c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f267415f2a0 0x7f26741ea120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35420/0 (socket says 172.21.15.37:35420) 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267994c640 1 -- 172.21.15.37:0/3233456185 learned_addr learned my addr 172.21.15.37:0/3233456185 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267894a640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26740d0060 0x7f26741e9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267914b640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f267406e360 0x7f2674159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267994c640 1 -- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26740d0060 msgr2=0x7f26741e9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267994c640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26740d0060 0x7f26741e9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267994c640 1 -- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f267406e360 msgr2=0x7f2674159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267994c640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f267406e360 0x7f2674159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267994c640 1 -- 172.21.15.37:0/3233456185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26741e8640 con 0x7f267415f2a0 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267994c640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f267415f2a0 0x7f26741ea120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f266402acc0 tx=0x7f2664004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f2671ffb640 1 -- 172.21.15.37:0/3233456185 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664040070 con 0x7f267415f2a0 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f2671ffb640 1 -- 172.21.15.37:0/3233456185 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2664031c40 con 0x7f267415f2a0 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f2671ffb640 1 -- 172.21.15.37:0/3233456185 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664009450 con 0x7f267415f2a0 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.637+0000 7f267b3af640 1 -- 172.21.15.37:0/3233456185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26741e8920 con 0x7f267415f2a0 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.638+0000 7f267b3af640 1 -- 172.21.15.37:0/3233456185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26741e8ee0 con 0x7f267415f2a0 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.638+0000 7f267b3af640 1 -- 172.21.15.37:0/3233456185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26740693e0 con 0x7f267415f2a0 2024-04-26T20:22:03.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.639+0000 7f2671ffb640 1 -- 172.21.15.37:0/3233456185 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2664039040 con 0x7f267415f2a0 2024-04-26T20:22:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.639+0000 7f2671ffb640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f266006c8f0 0x7f266006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.639+0000 7f2671ffb640 1 -- 172.21.15.37:0/3233456185 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f26640ad1c0 con 0x7f267415f2a0 2024-04-26T20:22:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.639+0000 7f267914b640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f266006c8f0 0x7f266006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.640+0000 7f267914b640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f266006c8f0 0x7f266006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f266c007fd0 tx=0x7f266c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:03.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.644+0000 7f2671ffb640 1 -- 172.21.15.37:0/3233456185 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f266407afa0 con 0x7f267415f2a0 2024-04-26T20:22:03.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.833+0000 7f267b3af640 1 -- 172.21.15.37:0/3233456185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2674076170 con 0x7f267415f2a0 2024-04-26T20:22:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.834+0000 7f2671ffb640 1 -- 172.21.15.37:0/3233456185 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f266407f2a0 con 0x7f267415f2a0 2024-04-26T20:22:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:03.835 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:03.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 -- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f266006c8f0 msgr2=0x7f266006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f266006c8f0 0x7f266006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f266c007fd0 tx=0x7f266c009040 comp rx=0 tx=0).stop 2024-04-26T20:22:03.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 -- 172.21.15.37:0/3233456185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f267415f2a0 msgr2=0x7f26741ea120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f267415f2a0 0x7f26741ea120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f266402acc0 tx=0x7f2664004a40 comp rx=0 tx=0).stop 2024-04-26T20:22:03.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 -- 172.21.15.37:0/3233456185 shutdown_connections 2024-04-26T20:22:03.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f267415f2a0 0x7f26741ea120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f266006c8f0 0x7f266006edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26740d0060 0x7f26741e9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 --2- 172.21.15.37:0/3233456185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f267406e360 0x7f2674159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 -- 172.21.15.37:0/3233456185 >> 172.21.15.37:0/3233456185 conn(0x7f267409ba50 msgr2=0x7f2674160b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:03.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 -- 172.21.15.37:0/3233456185 shutdown_connections 2024-04-26T20:22:03.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.837+0000 7f267b3af640 1 -- 172.21.15.37:0/3233456185 wait complete. 2024-04-26T20:22:03.848 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:22:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.948+0000 7fdd9d83f640 1 Processor -- start 2024-04-26T20:22:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.948+0000 7fdd9d83f640 1 -- start start 2024-04-26T20:22:03.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.949+0000 7fdd9d83f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdd98145790 0x7fdd98070d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.949+0000 7fdd9d83f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdd980700a0 0x7fdd980b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.949+0000 7fdd9d83f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd980b4f10 con 0x7fdd980b64f0 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.949+0000 7fdd9d83f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fdd980b51f0 con 0x7fdd98145790 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.949+0000 7fdd9d83f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fdd980b54d0 con 0x7fdd980700a0 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.949+0000 7fdd977fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdd980b64f0 0x7fdd980730d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44434/0 (socket says 172.21.15.37:44434) 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.949+0000 7fdd96ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdd98145790 0x7fdd98070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.949+0000 7fdd977fe640 1 -- 172.21.15.37:0/2650435212 learned_addr learned my addr 172.21.15.37:0/2650435212 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.949+0000 7fdd967fc640 1 --2- 172.21.15.37:0/2650435212 >> v2:172.21.15.134:3300/0 conn(0x7fdd980700a0 0x7fdd980b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.950+0000 7fdd95ffb640 1 -- 172.21.15.37:0/2650435212 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2901834782 0 0) 0x7fdd980b4f10 con 0x7fdd980b64f0 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.950+0000 7fdd95ffb640 1 -- 172.21.15.37:0/2650435212 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd80003520 con 0x7fdd980b64f0 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.950+0000 7fdd96ffd640 1 -- 172.21.15.37:0/2650435212 >> v2:172.21.15.134:3300/0 conn(0x7fdd980700a0 msgr2=0x7fdd980b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.950+0000 7fdd96ffd640 1 --2- 172.21.15.37:0/2650435212 >> v2:172.21.15.134:3300/0 conn(0x7fdd980700a0 0x7fdd980b5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.950+0000 7fdd96ffd640 1 -- 172.21.15.37:0/2650435212 >> v1:172.21.15.37:6789/0 conn(0x7fdd980b64f0 legacy=0x7fdd980730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.950+0000 7fdd96ffd640 1 -- 172.21.15.37:0/2650435212 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd980b57b0 con 0x7fdd98145790 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.950+0000 7fdd96ffd640 1 --2- 172.21.15.37:0/2650435212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd98145790 0x7fdd98070d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd98074ae0 tx=0x7fdd9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af88fa7a55c7d828 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.950+0000 7fdd95ffb640 1 -- 172.21.15.37:0/2650435212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90037070 con 0x7fdd98145790 2024-04-26T20:22:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.950+0000 7fdd95ffb640 1 -- 172.21.15.37:0/2650435212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd90032440 con 0x7fdd98145790 2024-04-26T20:22:03.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.950+0000 7fdd95ffb640 1 -- 172.21.15.37:0/2650435212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90032810 con 0x7fdd98145790 2024-04-26T20:22:03.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.951+0000 7fdd9d83f640 1 -- 172.21.15.37:0/2650435212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd98145790 msgr2=0x7fdd98070d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.951+0000 7fdd9d83f640 1 --2- 172.21.15.37:0/2650435212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd98145790 0x7fdd98070d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd98074ae0 tx=0x7fdd9002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.951+0000 7fdd9d83f640 1 -- 172.21.15.37:0/2650435212 shutdown_connections 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.951+0000 7fdd9d83f640 1 --2- 172.21.15.37:0/2650435212 >> v2:172.21.15.134:3300/0 conn(0x7fdd980700a0 0x7fdd980b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.951+0000 7fdd9d83f640 1 --2- 172.21.15.37:0/2650435212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd98145790 0x7fdd98070d80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.951+0000 7fdd9d83f640 1 -- 172.21.15.37:0/2650435212 >> 172.21.15.37:0/2650435212 conn(0x7fdd980b3b70 msgr2=0x7fdd980b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.951+0000 7fdd9d83f640 1 -- 172.21.15.37:0/2650435212 shutdown_connections 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.951+0000 7fdd9d83f640 1 -- 172.21.15.37:0/2650435212 wait complete. 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.952+0000 7fdd9d83f640 1 Processor -- start 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.952+0000 7fdd9d83f640 1 -- start start 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.953+0000 7fdd9d83f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd980700a0 0x7fdd9815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.953+0000 7fdd9d83f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd98145790 0x7fdd9813d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.953+0000 7fdd9d83f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd9813da00 0x7fdd980d2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.953+0000 7fdd9d83f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98071440 con 0x7fdd98145790 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.953+0000 7fdd9d83f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd980712c0 con 0x7fdd980700a0 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.953+0000 7fdd9d83f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdd980737e0 con 0x7fdd9813da00 2024-04-26T20:22:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.953+0000 7fdd967fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd98145790 0x7fdd9813d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.953+0000 7fdd967fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd98145790 0x7fdd9813d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35450/0 (socket says 172.21.15.37:35450) 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.953+0000 7fdd967fc640 1 -- 172.21.15.37:0/114901617 learned_addr learned my addr 172.21.15.37:0/114901617 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.953+0000 7fdd96ffd640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd980700a0 0x7fdd9815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd977fe640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd9813da00 0x7fdd980d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd967fc640 1 -- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd9813da00 msgr2=0x7fdd980d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd967fc640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd9813da00 0x7fdd980d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd967fc640 1 -- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd980700a0 msgr2=0x7fdd9815b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd967fc640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd980700a0 0x7fdd9815b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd967fc640 1 -- 172.21.15.37:0/114901617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9813ea10 con 0x7fdd98145790 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd96ffd640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd980700a0 0x7fdd9815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd977fe640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd9813da00 0x7fdd980d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd967fc640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd98145790 0x7fdd9813d4c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00e980 tx=0x7fdd8c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd7ffff640 1 -- 172.21.15.37:0/114901617 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c017070 con 0x7fdd98145790 2024-04-26T20:22:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd7ffff640 1 -- 172.21.15.37:0/114901617 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8c00c9a0 con 0x7fdd98145790 2024-04-26T20:22:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd9d83f640 1 -- 172.21.15.37:0/114901617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9813ecf0 con 0x7fdd98145790 2024-04-26T20:22:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd7ffff640 1 -- 172.21.15.37:0/114901617 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c00cd70 con 0x7fdd98145790 2024-04-26T20:22:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.954+0000 7fdd9d83f640 1 -- 172.21.15.37:0/114901617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9813f2b0 con 0x7fdd98145790 2024-04-26T20:22:03.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.955+0000 7fdd9d83f640 1 -- 172.21.15.37:0/114901617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd9813de10 con 0x7fdd98145790 2024-04-26T20:22:03.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.955+0000 7fdd7ffff640 1 -- 172.21.15.37:0/114901617 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd8c01a070 con 0x7fdd98145790 2024-04-26T20:22:03.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.956+0000 7fdd7ffff640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd8406c920 0x7fdd8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.956+0000 7fdd7ffff640 1 -- 172.21.15.37:0/114901617 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdd8c08bde0 con 0x7fdd98145790 2024-04-26T20:22:03.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.956+0000 7fdd96ffd640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd8406c920 0x7fdd8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.957+0000 7fdd96ffd640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd8406c920 0x7fdd8406ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdd90034000 tx=0x7fdd9002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:03.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:03.960+0000 7fdd7ffff640 1 -- 172.21.15.37:0/114901617 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd9813de10 con 0x7fdd98145790 2024-04-26T20:22:04.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.183+0000 7fdd9d83f640 1 -- 172.21.15.37:0/114901617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd980693e0 con 0x7fdd98145790 2024-04-26T20:22:04.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.184+0000 7fdd7ffff640 1 -- 172.21.15.37:0/114901617 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7fdd8c05a210 con 0x7fdd98145790 2024-04-26T20:22:04.184 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:04.185 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2911741012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2911741012},{"type":"v1","addr":"172.21.15.134:6836","nonce":2911741012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1600483467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1600483467},{"type":"v1","addr":"172.21.15.37:6835","nonce":1600483467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2722614558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2722614558},{"type":"v1","addr":"172.21.15.134:6837","nonce":2722614558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:21:49.237855+0000","modified":"2024-04-26T20:21:52.255148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3920647125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3920647125},{"type":"v1","addr":"172.21.15.37:6834","nonce":3920647125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-04-26T20:22:04.185 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-26T20:22:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.185+0000 7fdd9d83f640 1 -- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd8406c920 msgr2=0x7fdd8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.185+0000 7fdd9d83f640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd8406c920 0x7fdd8406ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdd90034000 tx=0x7fdd9002e000 comp rx=0 tx=0).stop 2024-04-26T20:22:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.185+0000 7fdd9d83f640 1 -- 172.21.15.37:0/114901617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd98145790 msgr2=0x7fdd9813d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.185+0000 7fdd9d83f640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd98145790 0x7fdd9813d4c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00e980 tx=0x7fdd8c00ee40 comp rx=0 tx=0).stop 2024-04-26T20:22:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.186+0000 7fdd9d83f640 1 -- 172.21.15.37:0/114901617 shutdown_connections 2024-04-26T20:22:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.186+0000 7fdd9d83f640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd8406c920 0x7fdd8406ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.186+0000 7fdd9d83f640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd9813da00 0x7fdd980d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.186+0000 7fdd9d83f640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd98145790 0x7fdd9813d4c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.186+0000 7fdd9d83f640 1 --2- 172.21.15.37:0/114901617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd980700a0 0x7fdd9815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.186+0000 7fdd9d83f640 1 -- 172.21.15.37:0/114901617 >> 172.21.15.37:0/114901617 conn(0x7fdd980b3b70 msgr2=0x7fdd980b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.186+0000 7fdd9d83f640 1 -- 172.21.15.37:0/114901617 shutdown_connections 2024-04-26T20:22:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.186+0000 7fdd9d83f640 1 -- 172.21.15.37:0/114901617 wait complete. 2024-04-26T20:22:04.599 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:21:49.237855+0000', 'modified': '2024-04-26T20:21:52.255148+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6834/3920647125', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 3920647125}, {'type': 'v1', 'addr': '172.21.15.37:6834', 'nonce': 3920647125}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4284, 'qdb_cluster': [4284]} 2024-04-26T20:22:04.599 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:22:04.599 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T20:22:04.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.705+0000 7f6eae11c640 1 Processor -- start 2024-04-26T20:22:04.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.705+0000 7f6eae11c640 1 -- start start 2024-04-26T20:22:04.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6eae11c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6ea8162e40 0x7f6ea80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6eae11c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6ea80cff40 0x7f6ea80ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6eae11c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ea80b1d70 con 0x7f6ea80b2050 2024-04-26T20:22:04.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6eae11c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea8163220 con 0x7f6ea8162e40 2024-04-26T20:22:04.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6eae11c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea80d0eb0 con 0x7f6ea80cff40 2024-04-26T20:22:04.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6ea77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6ea8162e40 0x7f6ea80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6ea77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6ea8162e40 0x7f6ea80d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35456/0 (socket says 172.21.15.37:35456) 2024-04-26T20:22:04.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6ea77fe640 1 -- 172.21.15.37:0/2104936948 learned_addr learned my addr 172.21.15.37:0/2104936948 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:04.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6ea77fe640 1 -- 172.21.15.37:0/2104936948 >> v2:172.21.15.134:3300/0 conn(0x7f6ea80cff40 msgr2=0x7f6ea80ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6ea7fff640 1 --2- 172.21.15.37:0/2104936948 >> v2:172.21.15.134:3300/0 conn(0x7f6ea80cff40 0x7f6ea80ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/2104936948 >> v2:172.21.15.134:3300/0 conn(0x7f6ea80cff40 0x7f6ea80ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6ea77fe640 1 -- 172.21.15.37:0/2104936948 >> v1:172.21.15.134:6789/0 conn(0x7f6ea80b2050 legacy=0x7f6ea80cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6ea77fe640 1 -- 172.21.15.37:0/2104936948 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea80d1960 con 0x7f6ea8162e40 2024-04-26T20:22:04.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.706+0000 7f6ea7fff640 1 --2- 172.21.15.37:0/2104936948 >> v2:172.21.15.134:3300/0 conn(0x7f6ea80cff40 0x7f6ea80ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:04.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.707+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/2104936948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ea8162e40 0x7f6ea80d1340 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8073470 tx=0x7f6e9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33c903ead5568ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:04.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.707+0000 7f6ea67fc640 1 -- 172.21.15.37:0/2104936948 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98037070 con 0x7f6ea8162e40 2024-04-26T20:22:04.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.707+0000 7f6ea67fc640 1 -- 172.21.15.37:0/2104936948 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e98032440 con 0x7f6ea8162e40 2024-04-26T20:22:04.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.707+0000 7f6ea67fc640 1 -- 172.21.15.37:0/2104936948 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98032810 con 0x7f6ea8162e40 2024-04-26T20:22:04.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.707+0000 7f6eae11c640 1 -- 172.21.15.37:0/2104936948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ea8162e40 msgr2=0x7f6ea80d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.707+0000 7f6eae11c640 1 --2- 172.21.15.37:0/2104936948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ea8162e40 0x7f6ea80d1340 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8073470 tx=0x7f6e9802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.708+0000 7f6eae11c640 1 -- 172.21.15.37:0/2104936948 shutdown_connections 2024-04-26T20:22:04.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.708+0000 7f6eae11c640 1 --2- 172.21.15.37:0/2104936948 >> v2:172.21.15.134:3300/0 conn(0x7f6ea80cff40 0x7f6ea80ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.708+0000 7f6eae11c640 1 --2- 172.21.15.37:0/2104936948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ea8162e40 0x7f6ea80d1340 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.708+0000 7f6eae11c640 1 -- 172.21.15.37:0/2104936948 >> 172.21.15.37:0/2104936948 conn(0x7f6ea809ba50 msgr2=0x7f6ea809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:04.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.708+0000 7f6eae11c640 1 -- 172.21.15.37:0/2104936948 shutdown_connections 2024-04-26T20:22:04.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.708+0000 7f6eae11c640 1 -- 172.21.15.37:0/2104936948 wait complete. 2024-04-26T20:22:04.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.709+0000 7f6eae11c640 1 Processor -- start 2024-04-26T20:22:04.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.709+0000 7f6eae11c640 1 -- start start 2024-04-26T20:22:04.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6eae11c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ea80b2050 0x7f6ea81602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6eae11c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea80cff40 0x7f6ea81f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6eae11c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ea8162e40 0x7f6ea81f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6eae11c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80b24e0 con 0x7f6ea8162e40 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6eae11c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80d1f20 con 0x7f6ea80cff40 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6eae11c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea806ea40 con 0x7f6ea80b2050 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ea8162e40 0x7f6ea81f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ea8162e40 0x7f6ea81f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35464/0 (socket says 172.21.15.37:35464) 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea7fff640 1 -- 172.21.15.37:0/3279108620 learned_addr learned my addr 172.21.15.37:0/3279108620 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ea80b2050 0x7f6ea81602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea80cff40 0x7f6ea81f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea7fff640 1 -- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ea80b2050 msgr2=0x7f6ea81602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea7fff640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ea80b2050 0x7f6ea81602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea7fff640 1 -- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea80cff40 msgr2=0x7f6ea81f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea7fff640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea80cff40 0x7f6ea81f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea7fff640 1 -- 172.21.15.37:0/3279108620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea81f6120 con 0x7f6ea8162e40 2024-04-26T20:22:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ea80b2050 0x7f6ea81602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea80cff40 0x7f6ea81f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6ea7fff640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ea8162e40 0x7f6ea81f4a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000cef0 tx=0x7f6ea000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6e8ffff640 1 -- 172.21.15.37:0/3279108620 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea001a070 con 0x7f6ea8162e40 2024-04-26T20:22:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6e8ffff640 1 -- 172.21.15.37:0/3279108620 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea000d860 con 0x7f6ea8162e40 2024-04-26T20:22:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6eae11c640 1 -- 172.21.15.37:0/3279108620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea81608f0 con 0x7f6ea8162e40 2024-04-26T20:22:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.710+0000 7f6e8ffff640 1 -- 172.21.15.37:0/3279108620 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea000dc50 con 0x7f6ea8162e40 2024-04-26T20:22:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.711+0000 7f6eae11c640 1 -- 172.21.15.37:0/3279108620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea8160eb0 con 0x7f6ea8162e40 2024-04-26T20:22:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.712+0000 7f6e8ffff640 1 -- 172.21.15.37:0/3279108620 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ea001d070 con 0x7f6ea8162e40 2024-04-26T20:22:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.713+0000 7f6e8ffff640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6e9406c8f0 0x7f6e9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.713+0000 7f6e8ffff640 1 -- 172.21.15.37:0/3279108620 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6ea008c3e0 con 0x7f6ea8162e40 2024-04-26T20:22:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.713+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6e9406c8f0 0x7f6e9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.714+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6e9406c8f0 0x7f6e9406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e98034000 tx=0x7f6e9802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.714+0000 7f6eb0e31740 1 Processor -- start 2024-04-26T20:22:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.714+0000 7f6eb0e31740 1 -- start start 2024-04-26T20:22:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad38f440 0x5581ad38f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad365120 0x5581ad3ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5581ad2c9740 con 0x5581ad3acce0 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5581ad2653a0 con 0x5581ad38f440 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5581ad2c5700 con 0x5581ad365120 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35466/0 (socket says 172.21.15.37:35466) 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/2850401247 learned_addr learned my addr 172.21.15.37:0/2850401247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/2850401247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad38f440 0x5581ad38f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6ea7fff640 1 --2- 172.21.15.37:0/2850401247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad365120 0x5581ad3ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/2850401247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad365120 msgr2=0x5581ad3ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/2850401247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad365120 0x5581ad3ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/2850401247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad38f440 msgr2=0x5581ad38f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/2850401247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad38f440 0x5581ad38f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/2850401247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ad281750 con 0x5581ad3acce0 2024-04-26T20:22:04.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/2850401247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ad230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c001650 tx=0x7f6e9c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffafccf6ded0e00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:04.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6e8d7fa640 1 -- 172.21.15.37:0/2850401247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c019070 con 0x5581ad3acce0 2024-04-26T20:22:04.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6e8d7fa640 1 -- 172.21.15.37:0/2850401247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9c0052e0 con 0x5581ad3acce0 2024-04-26T20:22:04.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.715+0000 7f6e8d7fa640 1 -- 172.21.15.37:0/2850401247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c007250 con 0x5581ad3acce0 2024-04-26T20:22:04.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.716+0000 7f6eb0e31740 1 -- 172.21.15.37:0/2850401247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 msgr2=0x5581ad3ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.716+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/2850401247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ad230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c001650 tx=0x7f6e9c001b10 comp rx=0 tx=0).stop 2024-04-26T20:22:04.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.716+0000 7f6eb0e31740 1 -- 172.21.15.37:0/2850401247 shutdown_connections 2024-04-26T20:22:04.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.716+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/2850401247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad365120 0x5581ad3ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.716+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/2850401247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ad230 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.716+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/2850401247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad38f440 0x5581ad38f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.716+0000 7f6eb0e31740 1 -- 172.21.15.37:0/2850401247 >> 172.21.15.37:0/2850401247 conn(0x5581ad361a50 msgr2=0x5581ad3aa250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.716+0000 7f6eb0e31740 1 -- 172.21.15.37:0/2850401247 shutdown_connections 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.716+0000 7f6eb0e31740 1 -- 172.21.15.37:0/2850401247 wait complete. 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.717+0000 7f6eb0e31740 1 -- start start 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.717+0000 7f6eb0e31740 1 Processor -- start 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.717+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad3be4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.717+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3acce0 0x5581ad3ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.717+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3bd5b0 0x5581ad3ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.717+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5581ad365ec0 con 0x5581ad3bd5b0 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.717+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5581ad250fe0 con 0x5581ad3acce0 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.717+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5581ad35f980 con 0x5581ad38f440 2024-04-26T20:22:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6eb0e31740 10 client.0 _ll_register_callbacks cb 0x5581ad267150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3bd5b0 0x5581ad3ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3bd5b0 0x5581ad3ab710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35480/0 (socket says 172.21.15.37:35480) 2024-04-26T20:22:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea7fff640 1 -- 172.21.15.37:0/720099013 learned_addr learned my addr 172.21.15.37:0/720099013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3acce0 0x5581ad3ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad3be4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea7fff640 1 -- 172.21.15.37:0/720099013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 msgr2=0x5581ad3be4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea7fff640 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad3be4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea7fff640 1 -- 172.21.15.37:0/720099013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3acce0 msgr2=0x5581ad3ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea7fff640 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3acce0 0x5581ad3ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea7fff640 1 -- 172.21.15.37:0/720099013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581acfdb2d0 con 0x5581ad3bd5b0 2024-04-26T20:22:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad3be4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:04.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6ea7fff640 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3bd5b0 0x5581ad3ab710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6ea005a160 tx=0x7f6ea005e580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9901893883abfc52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:04.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6e867fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T20:22:04.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6e867fc640 1 -- 172.21.15.37:0/720099013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea005e170 con 0x5581ad3bd5b0 2024-04-26T20:22:04.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6e867fc640 1 -- 172.21.15.37:0/720099013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6ea005e310 con 0x5581ad3bd5b0 2024-04-26T20:22:04.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6eb0e31740 1 -- 172.21.15.37:0/720099013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e9c0052e0 con 0x5581ad3bd5b0 2024-04-26T20:22:04.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6e867fc640 1 -- 172.21.15.37:0/720099013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f6ea0077050 con 0x5581ad3bd5b0 2024-04-26T20:22:04.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6eb0e31740 10 client.4665 fetch_fsmap learned FSMap version 7 2024-04-26T20:22:04.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6eb0e31740 1 -- 172.21.15.37:0/720099013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5581ad0a56b0 con 0x5581ad3bd5b0 2024-04-26T20:22:04.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6e867fc640 1 -- 172.21.15.37:0/720099013 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9c0052e0 con 0x5581ad3bd5b0 2024-04-26T20:22:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.718+0000 7f6e867fc640 1 -- 172.21.15.37:0/720099013 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f6ea0062920 con 0x5581ad3bd5b0 2024-04-26T20:22:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.719+0000 7f6eb0e31740 10 client.4665 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T20:22:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.719+0000 7f6eb0e31740 10 client.4665 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:22:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.719+0000 7f6eb0e31740 20 client.4665 populate_metadata read hostname 'smithi037' 2024-04-26T20:22:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.719+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.37:6832/3920647125,v1:172.21.15.37:6834/3920647125] conn(0x5581ad3ba790 0x5581ad3bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.719+0000 7f6eb0e31740 4 client.4665 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:22:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.719+0000 7f6eb0e31740 1 -- 172.21.15.37:0/720099013 --> [v2:172.21.15.37:6832/3920647125,v1:172.21.15.37:6834/3920647125] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581ad2d7350 con 0x5581ad3ba790 2024-04-26T20:22:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.719+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.37:6832/3920647125,v1:172.21.15.37:6834/3920647125] conn(0x5581ad3ba790 0x5581ad3bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.719+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.37:6832/3920647125,v1:172.21.15.37:6834/3920647125] conn(0x5581ad3ba790 0x5581ad3bab70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d8407e918382c85 server_cookie=88b88ee5f67b5845 in_seq=0 out_seq=0 2024-04-26T20:22:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.719+0000 7f6e867fc640 10 client.4665 ms_handle_connect on v2:172.21.15.37:6832/3920647125 2024-04-26T20:22:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6e867fc640 1 -- 172.21.15.37:0/720099013 <== mds.0 v2:172.21.15.37:6832/3920647125 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581ad2d7350 con 0x5581ad3ba790 2024-04-26T20:22:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6e867fc640 10 client.4665 handle_command_reply: tid=0 2024-04-26T20:22:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 1 client.4665 shutdown 2024-04-26T20:22:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 1 -- 172.21.15.37:0/720099013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3bd5b0 msgr2=0x5581ad3ab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3bd5b0 0x5581ad3ab710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6ea005a160 tx=0x7f6ea005e580 comp rx=0 tx=0).stop 2024-04-26T20:22:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 1 -- 172.21.15.37:0/720099013 shutdown_connections 2024-04-26T20:22:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.37:6832/3920647125,v1:172.21.15.37:6834/3920647125] conn(0x5581ad3ba790 0x5581ad3bab70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3bd5b0 0x5581ad3ab710 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3acce0 0x5581ad3ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6e867fc640 0 client.4665 ms_handle_reset on v2:172.21.15.37:6832/3920647125 2024-04-26T20:22:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/720099013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad3be4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 1 -- 172.21.15.37:0/720099013 >> 172.21.15.37:0/720099013 conn(0x5581ad365120 msgr2=0x5581ad3ad7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 1 -- 172.21.15.37:0/720099013 shutdown_connections 2024-04-26T20:22:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 1 -- 172.21.15.37:0/720099013 wait complete. 2024-04-26T20:22:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.720+0000 7f6eb0e31740 20 client.4665 trim_cache size 0 max 16384 2024-04-26T20:22:04.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.738+0000 7f6eb0e31740 1 Processor -- start 2024-04-26T20:22:04.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.738+0000 7f6eb0e31740 1 -- start start 2024-04-26T20:22:04.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad35acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ab280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3bd5b0 0x5581ad3ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80b81c0 con 0x5581ad3acce0 2024-04-26T20:22:04.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80c7550 con 0x5581ad3bd5b0 2024-04-26T20:22:04.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea80b7b70 con 0x5581ad38f440 2024-04-26T20:22:04.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ab280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35496/0 (socket says 172.21.15.37:35496) 2024-04-26T20:22:04.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/470255497 learned_addr learned my addr 172.21.15.37:0/470255497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:04.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/470255497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 msgr2=0x5581ad35acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/470255497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad35acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/470255497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3bd5b0 msgr2=0x5581ad3ab7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/470255497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3bd5b0 0x5581ad3ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/470255497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ad3bd3c0 con 0x5581ad3acce0 2024-04-26T20:22:04.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.739+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/470255497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ab280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c001570 tx=0x7f6e9c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785219ddcaf9d7d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:04.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6e8cff9640 1 -- 172.21.15.37:0/470255497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c019040 con 0x5581ad3acce0 2024-04-26T20:22:04.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6e8cff9640 1 -- 172.21.15.37:0/470255497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9c006790 con 0x5581ad3acce0 2024-04-26T20:22:04.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6e8cff9640 1 -- 172.21.15.37:0/470255497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c006b60 con 0x5581ad3acce0 2024-04-26T20:22:04.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6eb0e31740 1 -- 172.21.15.37:0/470255497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 msgr2=0x5581ad3ab280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/470255497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ab280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c001570 tx=0x7f6e9c00a000 comp rx=0 tx=0).stop 2024-04-26T20:22:04.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6eb0e31740 1 -- 172.21.15.37:0/470255497 shutdown_connections 2024-04-26T20:22:04.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/470255497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3bd5b0 0x5581ad3ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/470255497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad3ab280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/470255497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad35acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6eb0e31740 1 -- 172.21.15.37:0/470255497 >> 172.21.15.37:0/470255497 conn(0x5581ad365120 msgr2=0x5581ad364640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:04.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6eb0e31740 1 -- 172.21.15.37:0/470255497 shutdown_connections 2024-04-26T20:22:04.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.740+0000 7f6eb0e31740 1 -- 172.21.15.37:0/470255497 wait complete. 2024-04-26T20:22:04.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.741+0000 7f6eb0e31740 1 -- start start 2024-04-26T20:22:04.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.741+0000 7f6eb0e31740 1 Processor -- start 2024-04-26T20:22:04.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad41a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad39fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6eb0e31740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3bd5b0 0x5581ad3a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5581ad3a3be0 con 0x5581ad3acce0 2024-04-26T20:22:04.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5581ad393080 con 0x5581ad3bd5b0 2024-04-26T20:22:04.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6eb0e31740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5581ad3bbcb0 con 0x5581ad38f440 2024-04-26T20:22:04.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6eb0e31740 10 client.0 _ll_register_callbacks cb 0x7f6ea005a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:04.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad39fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad39fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35498/0 (socket says 172.21.15.37:35498) 2024-04-26T20:22:04.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/471740951 learned_addr learned my addr 172.21.15.37:0/471740951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:04.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6ea7fff640 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3bd5b0 0x5581ad3a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad41a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/471740951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 msgr2=0x5581ad41a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad41a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/471740951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3bd5b0 msgr2=0x5581ad3a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3bd5b0 0x5581ad3a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6ea6ffd640 1 -- 172.21.15.37:0/471740951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ad3c4a40 con 0x5581ad3acce0 2024-04-26T20:22:04.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6ea6ffd640 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad39fbe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c020040 tx=0x7f6e9c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc8cb88fc34b00d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:04.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6e867fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T20:22:04.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6e867fc640 1 -- 172.21.15.37:0/471740951 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c006940 con 0x5581ad3acce0 2024-04-26T20:22:04.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6e867fc640 1 -- 172.21.15.37:0/471740951 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6e9c006ae0 con 0x5581ad3acce0 2024-04-26T20:22:04.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.742+0000 7f6eb0e31740 1 -- 172.21.15.37:0/471740951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e9c006790 con 0x5581ad3acce0 2024-04-26T20:22:04.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6e867fc640 1 -- 172.21.15.37:0/471740951 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f6e9c034050 con 0x5581ad3acce0 2024-04-26T20:22:04.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6eb0e31740 10 client.4671 fetch_fsmap learned FSMap version 7 2024-04-26T20:22:04.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6eb0e31740 1 -- 172.21.15.37:0/471740951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5581ad3a4b70 con 0x5581ad3acce0 2024-04-26T20:22:04.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6e867fc640 1 -- 172.21.15.37:0/471740951 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9c006790 con 0x5581ad3acce0 2024-04-26T20:22:04.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6e867fc640 1 -- 172.21.15.37:0/471740951 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f6e9c037020 con 0x5581ad3acce0 2024-04-26T20:22:04.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6eb0e31740 10 client.4671 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T20:22:04.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6eb0e31740 10 client.4671 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:22:04.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6eb0e31740 20 client.4671 populate_metadata read hostname 'smithi037' 2024-04-26T20:22:04.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.37:6832/3920647125,v1:172.21.15.37:6834/3920647125] conn(0x5581ad3bdeb0 0x5581ad3be290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6eb0e31740 4 client.4671 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6eb0e31740 1 -- 172.21.15.37:0/471740951 --> [v2:172.21.15.37:6832/3920647125,v1:172.21.15.37:6834/3920647125] -- command(tid 0: {"prefix": "status"}) -- 0x5581ad2d11c0 con 0x5581ad3bdeb0 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.37:6832/3920647125,v1:172.21.15.37:6834/3920647125] conn(0x5581ad3bdeb0 0x5581ad3be290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6e867fc640 10 client.4671 ms_handle_connect on v2:172.21.15.37:6832/3920647125 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6ea77fe640 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.37:6832/3920647125,v1:172.21.15.37:6834/3920647125] conn(0x5581ad3bdeb0 0x5581ad3be290 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a3ea79865e14138 server_cookie=c4af2192f8d147bb in_seq=0 out_seq=0 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6e867fc640 1 -- 172.21.15.37:0/471740951 <== mds.0 v2:172.21.15.37:6832/3920647125 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5581ad2d11c0 con 0x5581ad3bdeb0 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.743+0000 7f6e867fc640 10 client.4671 handle_command_reply: tid=0 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 1 client.4671 shutdown 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 1 -- 172.21.15.37:0/471740951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 msgr2=0x5581ad39fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad39fbe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c020040 tx=0x7f6e9c011000 comp rx=0 tx=0).stop 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 1 -- 172.21.15.37:0/471740951 shutdown_connections 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.37:6832/3920647125,v1:172.21.15.37:6834/3920647125] conn(0x5581ad3bdeb0 0x5581ad3be290 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581ad3bd5b0 0x5581ad3a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ad3acce0 0x5581ad39fbe0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6e867fc640 0 client.4671 ms_handle_reset on v2:172.21.15.37:6832/3920647125 2024-04-26T20:22:04.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 1 --2- 172.21.15.37:0/471740951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5581ad38f440 0x5581ad41a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 1 -- 172.21.15.37:0/471740951 >> 172.21.15.37:0/471740951 conn(0x5581ad365120 msgr2=0x5581ad3bab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:04.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 1 -- 172.21.15.37:0/471740951 shutdown_connections 2024-04-26T20:22:04.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 1 -- 172.21.15.37:0/471740951 wait complete. 2024-04-26T20:22:04.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eb0e31740 20 client.4671 trim_cache size 0 max 16384 2024-04-26T20:22:04.752 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:22:04.752 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "d49b1c18-e1bf-49f2-acda-0904fa7c6684", 2024-04-26T20:22:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-26T20:22:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4284, 2024-04-26T20:22:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T20:22:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T20:22:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T20:22:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 14.861852619, 2024-04-26T20:22:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 7, 2024-04-26T20:22:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 20, 2024-04-26T20:22:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 20, 2024-04-26T20:22:04.754 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 21.790744177000001 2024-04-26T20:22:04.754 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:22:04.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eae11c640 1 -- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6e9406c8f0 msgr2=0x7f6e9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eae11c640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6e9406c8f0 0x7f6e9406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e98034000 tx=0x7f6e9802e000 comp rx=0 tx=0).stop 2024-04-26T20:22:04.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eae11c640 1 -- 172.21.15.37:0/3279108620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ea8162e40 msgr2=0x7f6ea81f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eae11c640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ea8162e40 0x7f6ea81f4a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000cef0 tx=0x7f6ea000a710 comp rx=0 tx=0).stop 2024-04-26T20:22:04.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eae11c640 1 -- 172.21.15.37:0/3279108620 shutdown_connections 2024-04-26T20:22:04.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eae11c640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ea8162e40 0x7f6ea81f4a00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eae11c640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea80cff40 0x7f6ea81f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eae11c640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6e9406c8f0 0x7f6e9406edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eae11c640 1 --2- 172.21.15.37:0/3279108620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ea80b2050 0x7f6ea81602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.744+0000 7f6eae11c640 1 -- 172.21.15.37:0/3279108620 >> 172.21.15.37:0/3279108620 conn(0x7f6ea809ba50 msgr2=0x7f6ea80b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:04.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.745+0000 7f6eae11c640 1 -- 172.21.15.37:0/3279108620 shutdown_connections 2024-04-26T20:22:04.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:04.745+0000 7f6eae11c640 1 -- 172.21.15.37:0/3279108620 wait complete. 2024-04-26T20:22:04.757 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T20:22:04.766 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T20:22:04.766 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T20:22:04.767 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T20:22:04.767 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T20:22:04.817 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T20:22:04.817 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T20:22:04.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:04.817 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T20:22:04.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:04.833 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-26T20:22:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-04-26T20:22:04.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:22:04.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:04.900 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T20:22:04.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:04.956 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-26T20:22:05.022 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-04-26T20:22:05.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:22:05.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:05.024 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-26T20:22:05.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:05.042 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-04-26T20:22:05.112 INFO:teuthology.orchestra.run.smithi134.stderr:Cannot find device "ceph-brx" 2024-04-26T20:22:05.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:22:05.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:05.114 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-26T20:22:05.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:05.172 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-04-26T20:22:05.239 INFO:teuthology.orchestra.run.smithi134.stderr:Cannot find device "ceph-brx" 2024-04-26T20:22:05.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:22:05.240 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-26T20:22:05.241 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:22:05.241 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:22:05.241 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:22:05.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.342+0000 7f3e48c74640 1 Processor -- start 2024-04-26T20:22:05.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.342+0000 7f3e48c74640 1 -- start start 2024-04-26T20:22:05.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.343+0000 7f3e48c74640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3e440712f0 0x7f3e440b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:05.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.343+0000 7f3e48c74640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e44070250 con 0x7f3e44073800 2024-04-26T20:22:05.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.343+0000 7f3e48c74640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e44070530 con 0x7f3e44144c50 2024-04-26T20:22:05.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.343+0000 7f3e48c74640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3e44070810 con 0x7f3e440712f0 2024-04-26T20:22:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.343+0000 7f3e42d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3e440712f0 0x7f3e440b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.343+0000 7f3e41d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3e44073800 0x7f3e440cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44448/0 (socket says 172.21.15.37:44448) 2024-04-26T20:22:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.343+0000 7f3e41d9b640 1 -- 172.21.15.37:0/2899885824 learned_addr learned my addr 172.21.15.37:0/2899885824 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.344+0000 7f3e42d9d640 1 -- 172.21.15.37:0/2899885824 >> v1:172.21.15.134:6789/0 conn(0x7f3e44144c50 legacy=0x7f3e440730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.344+0000 7f3e42d9d640 1 -- 172.21.15.37:0/2899885824 >> v1:172.21.15.37:6789/0 conn(0x7f3e44073800 legacy=0x7f3e440cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.344+0000 7f3e4159a640 1 -- 172.21.15.37:0/2899885824 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3106891933 0 0) 0x7f3e44070250 con 0x7f3e44073800 2024-04-26T20:22:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.344+0000 7f3e42d9d640 1 -- 172.21.15.37:0/2899885824 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e44070af0 con 0x7f3e440712f0 2024-04-26T20:22:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.344+0000 7f3e42d9d640 1 --2- 172.21.15.37:0/2899885824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e440712f0 0x7f3e440b6770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400bfa0 tx=0x7f3e3402c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972e4295df7467fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.344+0000 7f3e4159a640 1 -- 172.21.15.37:0/2899885824 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34009540 con 0x7f3e440712f0 2024-04-26T20:22:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.345+0000 7f3e4159a640 1 -- 172.21.15.37:0/2899885824 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e340096e0 con 0x7f3e440712f0 2024-04-26T20:22:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.345+0000 7f3e4159a640 1 -- 172.21.15.37:0/2899885824 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34009ab0 con 0x7f3e440712f0 2024-04-26T20:22:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.345+0000 7f3e48c74640 1 -- 172.21.15.37:0/2899885824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e440712f0 msgr2=0x7f3e440b6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.345+0000 7f3e48c74640 1 --2- 172.21.15.37:0/2899885824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e440712f0 0x7f3e440b6770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400bfa0 tx=0x7f3e3402c830 comp rx=0 tx=0).stop 2024-04-26T20:22:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.345+0000 7f3e48c74640 1 -- 172.21.15.37:0/2899885824 shutdown_connections 2024-04-26T20:22:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.345+0000 7f3e48c74640 1 --2- 172.21.15.37:0/2899885824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e440712f0 0x7f3e440b6770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.345+0000 7f3e48c74640 1 -- 172.21.15.37:0/2899885824 >> 172.21.15.37:0/2899885824 conn(0x7f3e440b3b00 msgr2=0x7f3e440b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.345+0000 7f3e48c74640 1 -- 172.21.15.37:0/2899885824 shutdown_connections 2024-04-26T20:22:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.345+0000 7f3e48c74640 1 -- 172.21.15.37:0/2899885824 wait complete. 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.346+0000 7f3e48c74640 1 Processor -- start 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.347+0000 7f3e48c74640 1 -- start start 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.347+0000 7f3e48c74640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e440712f0 0x7f3e4413f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.347+0000 7f3e48c74640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e44073800 0x7f3e4415e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.347+0000 7f3e48c74640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e44144c50 0x7f3e440c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.347+0000 7f3e48c74640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44145030 con 0x7f3e44144c50 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.347+0000 7f3e48c74640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44145b00 con 0x7f3e440712f0 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.347+0000 7f3e48c74640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3e441451b0 con 0x7f3e44073800 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.347+0000 7f3e42d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e44144c50 0x7f3e440c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.347+0000 7f3e42d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e44144c50 0x7f3e440c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35522/0 (socket says 172.21.15.37:35522) 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.347+0000 7f3e42d9d640 1 -- 172.21.15.37:0/606973935 learned_addr learned my addr 172.21.15.37:0/606973935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e41d9b640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e44073800 0x7f3e4415e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:05.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e4259c640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e440712f0 0x7f3e4413f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e42d9d640 1 -- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e44073800 msgr2=0x7f3e4415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e42d9d640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e44073800 0x7f3e4415e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e42d9d640 1 -- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e440712f0 msgr2=0x7f3e4413f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e42d9d640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e440712f0 0x7f3e4413f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e42d9d640 1 -- 172.21.15.37:0/606973935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e440c6d70 con 0x7f3e44144c50 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e4259c640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e440712f0 0x7f3e4413f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e42d9d640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e44144c50 0x7f3e440c9310 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400d110 tx=0x7f3e34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e3affd640 1 -- 172.21.15.37:0/606973935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34044070 con 0x7f3e44144c50 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e3affd640 1 -- 172.21.15.37:0/606973935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e340358e0 con 0x7f3e44144c50 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e48c74640 1 -- 172.21.15.37:0/606973935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4415e920 con 0x7f3e44144c50 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e3affd640 1 -- 172.21.15.37:0/606973935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34035cd0 con 0x7f3e44144c50 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.348+0000 7f3e48c74640 1 -- 172.21.15.37:0/606973935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4415eee0 con 0x7f3e44144c50 2024-04-26T20:22:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.349+0000 7f3e48c74640 1 -- 172.21.15.37:0/606973935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e440693e0 con 0x7f3e44144c50 2024-04-26T20:22:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.349+0000 7f3e3affd640 1 -- 172.21.15.37:0/606973935 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e34047070 con 0x7f3e44144c50 2024-04-26T20:22:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.350+0000 7f3e3affd640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e2406c8f0 0x7f3e2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.350+0000 7f3e3affd640 1 -- 172.21.15.37:0/606973935 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3e340aee70 con 0x7f3e44144c50 2024-04-26T20:22:05.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.350+0000 7f3e4259c640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e2406c8f0 0x7f3e2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:05.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.351+0000 7f3e4259c640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e2406c8f0 0x7f3e2406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00a830 tx=0x7f3e3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:05.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.354+0000 7f3e3affd640 1 -- 172.21.15.37:0/606973935 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e3407d450 con 0x7f3e44144c50 2024-04-26T20:22:05.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.579+0000 7f3e48c74640 1 -- 172.21.15.37:0/606973935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3e4406f470 con 0x7f3e44144c50 2024-04-26T20:22:05.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.579+0000 7f3e3affd640 1 -- 172.21.15.37:0/606973935 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f3e3407cd50 con 0x7f3e44144c50 2024-04-26T20:22:05.579 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:22:05.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 -- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e2406c8f0 msgr2=0x7f3e2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:05.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e2406c8f0 0x7f3e2406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00a830 tx=0x7f3e3c008040 comp rx=0 tx=0).stop 2024-04-26T20:22:05.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 -- 172.21.15.37:0/606973935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e44144c50 msgr2=0x7f3e440c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:05.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e44144c50 0x7f3e440c9310 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400d110 tx=0x7f3e34004a40 comp rx=0 tx=0).stop 2024-04-26T20:22:05.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 -- 172.21.15.37:0/606973935 shutdown_connections 2024-04-26T20:22:05.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e44144c50 0x7f3e440c9310 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e2406c8f0 0x7f3e2406edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e44073800 0x7f3e4415e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 --2- 172.21.15.37:0/606973935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e440712f0 0x7f3e4413f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 -- 172.21.15.37:0/606973935 >> 172.21.15.37:0/606973935 conn(0x7f3e440b3b00 msgr2=0x7f3e44143cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:05.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 -- 172.21.15.37:0/606973935 shutdown_connections 2024-04-26T20:22:05.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05.581+0000 7f3e48c74640 1 -- 172.21.15.37:0/606973935 wait complete. 2024-04-26T20:22:05.592 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:22:05.592 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:22:05.592 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-26T20:22:05.592 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:22:05.592 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:22:05.592 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:22:05.592 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:22:05.592 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:22:05.592 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:22:05.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:05.592 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-26T20:22:05.608 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:22:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:22:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:22:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-26T20:22:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:05.608 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-26T20:22:05.608 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-26T20:22:05.609 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:22:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-26T20:22:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T20:22:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42718sec preferred_lft 42718sec 2024-04-26T20:22:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-26T20:22:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:05.609 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-26T20:22:05.610 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:22:05.610 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:05.610 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T20:22:05.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:22:05.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-04-26T20:22:05.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-04-26T20:22:05.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:22:05.611 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T20:22:05.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:05.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:05.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:05.802 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:22:05.827 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-26T20:22:05.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:05.828 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-26T20:22:05.895 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-26T20:22:05.895 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:22:05.895 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:22:05.895 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:22:05.895 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:22:05.896 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:05.896 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T20:22:05.896 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:22:05.896 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:22:05.896 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:22:05.896 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T20:22:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:06.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:06.163 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T20:22:06.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:06.178 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-26T20:22:06.233 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:06.233 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T20:22:06.233 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:22:06.233 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T20:22:06.233 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T20:22:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:06.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:06.351 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T20:22:06.351 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:06.351 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T20:22:06.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T20:22:06.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:22:06.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T20:22:06.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T20:22:06.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T20:22:06.351 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T20:22:06.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:06.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:06.478 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:06.478 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T20:22:06.478 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-26T20:22:06.478 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:22:06.478 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T20:22:06.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:06.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:06.555 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:06.555 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:22:06.569 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:22:06.570 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:06.570 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:22:06.625 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T20:22:06.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:06.625 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:06.718 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:06.719 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:06.855 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:06.855 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:06.855 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-04-26T20:22:06.855 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with new device syntax: 0@d49b1c18-e1bf-49f2-acda-0904fa7c6684.cephfs=/ 2024-04-26T20:22:06.855 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.37:3300/172.21.15.134:3300/172.21.15.134:3301" will pass to kernel 2024-04-26T20:22:06.856 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T20:22:06.856 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:22:06.856 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:22:06.856 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:22:06.856 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:22:06.858 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:22:06.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:06.858 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:22:06.882 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:06.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:06.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:06.985 INFO:teuthology.orchestra.run.smithi037.stdout:client4683 2024-04-26T20:22:06.985 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:06.985 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:07.001 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:07.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:07.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:client4683 2024-04-26T20:22:07.102 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:07.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:07.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:07.163 INFO:teuthology.orchestra.run.smithi037.stdout:client4683 2024-04-26T20:22:07.163 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d49b1c18-e1bf-49f2-acda-0904fa7c6684.client4683/status') 2024-04-26T20:22:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:07.225 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.4683 (3)192.168.0.1:0/2407272723 2024-04-26T20:22:07.225 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-04-26T20:22:07.225 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-04-26T20:22:07.225 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-04-26T20:22:07.225 INFO:teuthology.orchestra.run.smithi037.stderr:69 bytes copied, 2.2998e-05 s, 3.0 MB/s 2024-04-26T20:22:07.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:07.232 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-26T20:22:07.232 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:22:07.233 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T20:22:07.233 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:22:07.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.333+0000 7f117866d640 1 Processor -- start 2024-04-26T20:22:07.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.333+0000 7f117866d640 1 -- start start 2024-04-26T20:22:07.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f117866d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f11700d22e0 0x7f11700b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f117866d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f117015f760 0x7f11700b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f117866d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11700d3de0 con 0x7f117015f380 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f117866d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f11700d43a0 con 0x7f11700d22e0 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f117866d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f11700d40c0 con 0x7f117015f760 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f1175c08640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f11700d22e0 0x7f11700b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f1176409640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f117015f380 0x7f11700d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34484/0 (socket says 172.21.15.37:34484) 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f1176409640 1 -- 172.21.15.37:0/3532603357 learned_addr learned my addr 172.21.15.37:0/3532603357 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f1176c0a640 1 --2- 172.21.15.37:0/3532603357 >> v2:172.21.15.134:3300/0 conn(0x7f117015f760 0x7f11700b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f1175c08640 1 -- 172.21.15.37:0/3532603357 >> v2:172.21.15.134:3300/0 conn(0x7f117015f760 msgr2=0x7f11700b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f1175c08640 1 --2- 172.21.15.37:0/3532603357 >> v2:172.21.15.134:3300/0 conn(0x7f117015f760 0x7f11700b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f1175c08640 1 -- 172.21.15.37:0/3532603357 >> v1:172.21.15.37:6789/0 conn(0x7f117015f380 legacy=0x7f11700d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f1175c08640 1 -- 172.21.15.37:0/3532603357 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11700b8280 con 0x7f11700d22e0 2024-04-26T20:22:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.334+0000 7f1175407640 1 -- 172.21.15.37:0/3532603357 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1372333286 0 0) 0x7f11700d3de0 con 0x7f117015f380 2024-04-26T20:22:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.335+0000 7f1175c08640 1 --2- 172.21.15.37:0/3532603357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11700d22e0 0x7f11700b8a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11640097a0 tx=0x7f116402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a9d757f84b9371 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.335+0000 7f1175407640 1 -- 172.21.15.37:0/3532603357 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164037070 con 0x7f11700d22e0 2024-04-26T20:22:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.335+0000 7f1175407640 1 -- 172.21.15.37:0/3532603357 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1164032440 con 0x7f11700d22e0 2024-04-26T20:22:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.335+0000 7f1175407640 1 -- 172.21.15.37:0/3532603357 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164032810 con 0x7f11700d22e0 2024-04-26T20:22:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.336+0000 7f117866d640 1 -- 172.21.15.37:0/3532603357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11700d22e0 msgr2=0x7f11700b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.336+0000 7f117866d640 1 --2- 172.21.15.37:0/3532603357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11700d22e0 0x7f11700b8a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11640097a0 tx=0x7f116402a270 comp rx=0 tx=0).stop 2024-04-26T20:22:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.336+0000 7f117866d640 1 -- 172.21.15.37:0/3532603357 shutdown_connections 2024-04-26T20:22:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.336+0000 7f117866d640 1 --2- 172.21.15.37:0/3532603357 >> v2:172.21.15.134:3300/0 conn(0x7f117015f760 0x7f11700b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.336+0000 7f117866d640 1 --2- 172.21.15.37:0/3532603357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11700d22e0 0x7f11700b8a80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.336+0000 7f117866d640 1 -- 172.21.15.37:0/3532603357 >> 172.21.15.37:0/3532603357 conn(0x7f117009e560 msgr2=0x7f11700d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.336+0000 7f117866d640 1 -- 172.21.15.37:0/3532603357 shutdown_connections 2024-04-26T20:22:07.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.336+0000 7f117866d640 1 -- 172.21.15.37:0/3532603357 wait complete. 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.337+0000 7f117866d640 1 Processor -- start 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.337+0000 7f117866d640 1 -- start start 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.338+0000 7f117866d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11700d22e0 0x7f1170154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.338+0000 7f117866d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117015f380 0x7f11700ba280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.338+0000 7f117866d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f117015f760 0x7f1170152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.338+0000 7f117866d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11700d1280 con 0x7f117015f380 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.338+0000 7f117866d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11700d1100 con 0x7f117015f760 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.338+0000 7f117866d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f11700d5b50 con 0x7f11700d22e0 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.338+0000 7f1175c08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117015f380 0x7f11700ba280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.338+0000 7f1175c08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117015f380 0x7f11700ba280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40096/0 (socket says 172.21.15.37:40096) 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.338+0000 7f1175c08640 1 -- 172.21.15.37:0/1521274006 learned_addr learned my addr 172.21.15.37:0/1521274006 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.338+0000 7f1176409640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11700d22e0 0x7f1170154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.338+0000 7f1176c0a640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f117015f760 0x7f1170152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.339+0000 7f1175c08640 1 -- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11700d22e0 msgr2=0x7f1170154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.339+0000 7f1175c08640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11700d22e0 0x7f1170154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.339+0000 7f1175c08640 1 -- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f117015f760 msgr2=0x7f1170152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.339+0000 7f1175c08640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f117015f760 0x7f1170152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.339+0000 7f1175c08640 1 -- 172.21.15.37:0/1521274006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1170153dc0 con 0x7f117015f380 2024-04-26T20:22:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.339+0000 7f1175c08640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117015f380 0x7f11700ba280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1164035040 tx=0x7f11640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.339+0000 7f116effd640 1 -- 172.21.15.37:0/1521274006 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164037050 con 0x7f117015f380 2024-04-26T20:22:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.339+0000 7f116effd640 1 -- 172.21.15.37:0/1521274006 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1164038040 con 0x7f117015f380 2024-04-26T20:22:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.339+0000 7f117866d640 1 -- 172.21.15.37:0/1521274006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11701540a0 con 0x7f117015f380 2024-04-26T20:22:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.339+0000 7f116effd640 1 -- 172.21.15.37:0/1521274006 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164004490 con 0x7f117015f380 2024-04-26T20:22:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.339+0000 7f117866d640 1 -- 172.21.15.37:0/1521274006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1170154660 con 0x7f117015f380 2024-04-26T20:22:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.340+0000 7f117866d640 1 -- 172.21.15.37:0/1521274006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117006a360 con 0x7f117015f380 2024-04-26T20:22:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.341+0000 7f116effd640 1 -- 172.21.15.37:0/1521274006 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f116403c6d0 con 0x7f117015f380 2024-04-26T20:22:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.341+0000 7f116effd640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f115c06c920 0x7f115c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.341+0000 7f116effd640 1 -- 172.21.15.37:0/1521274006 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f11640ac450 con 0x7f117015f380 2024-04-26T20:22:07.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.341+0000 7f1176409640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f115c06c920 0x7f115c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:07.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.342+0000 7f1176409640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f115c06c920 0x7f115c06ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f11600012e0 tx=0x7f1160001700 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.345+0000 7f116effd640 1 -- 172.21.15.37:0/1521274006 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f116407a230 con 0x7f117015f380 2024-04-26T20:22:07.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.569+0000 7f117866d640 1 -- 172.21.15.37:0/1521274006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f117015e9b0 con 0x7f117015f380 2024-04-26T20:22:07.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.570+0000 7f116effd640 1 -- 172.21.15.37:0/1521274006 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f116407e530 con 0x7f117015f380 2024-04-26T20:22:07.570 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:22:07.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 -- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f115c06c920 msgr2=0x7f115c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:07.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f115c06c920 0x7f115c06ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f11600012e0 tx=0x7f1160001700 comp rx=0 tx=0).stop 2024-04-26T20:22:07.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 -- 172.21.15.37:0/1521274006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117015f380 msgr2=0x7f11700ba280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:07.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117015f380 0x7f11700ba280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1164035040 tx=0x7f11640046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 -- 172.21.15.37:0/1521274006 shutdown_connections 2024-04-26T20:22:07.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f117015f760 0x7f1170152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117015f380 0x7f11700ba280 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f115c06c920 0x7f115c06ede0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 --2- 172.21.15.37:0/1521274006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11700d22e0 0x7f1170154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 -- 172.21.15.37:0/1521274006 >> 172.21.15.37:0/1521274006 conn(0x7f117009e560 msgr2=0x7f1170157c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:07.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 -- 172.21.15.37:0/1521274006 shutdown_connections 2024-04-26T20:22:07.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07.572+0000 7f117866d640 1 -- 172.21.15.37:0/1521274006 wait complete. 2024-04-26T20:22:07.583 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:22:07.583 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:22:07.583 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-26T20:22:07.583 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T20:22:07.583 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T20:22:07.583 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:22:07.583 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:22:07.583 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:22:07.583 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T20:22:07.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:07.584 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-26T20:22:07.599 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:22:07.599 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:22:07.599 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:22:07.599 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42716sec preferred_lft 42716sec 2024-04-26T20:22:07.600 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 06:a8:3c:e3:8e:a4 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::247e:2bff:fe9c:6859/64 scope link 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 06:a8:3c:e3:8e:a4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::4a8:3cff:fee3:8ea4/64 scope link tentative 2024-04-26T20:22:07.601 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:07.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:07.602 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:22:07.663 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-26T20:22:07.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:07.665 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-26T20:22:07.721 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-26T20:22:07.721 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:22:07.721 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:22:07.721 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:22:07.721 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:22:07.721 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:07.722 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T20:22:07.722 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:22:07.722 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:22:07.722 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:22:07.722 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T20:22:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:07.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:07.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:07.846 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T20:22:07.861 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:22:07.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:07.862 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-26T20:22:07.916 INFO:teuthology.orchestra.run.smithi037.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T20:22:07.917 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:07.917 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T20:22:07.917 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:22:07.917 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T20:22:07.917 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T20:22:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:08.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:08.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:08.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T20:22:08.058 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:22:08.058 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:22:08.058 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:22:08.058 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:08.058 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-26T20:22:08.058 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:08.058 INFO:teuthology.orchestra.run.smithi037.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:22:08.058 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 96:0a:b1:4b:cb:2e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:22:08.059 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:22:08.059 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:08.059 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::940a:b1ff:fe4b:cb2e/64 scope link tentative 2024-04-26T20:22:08.059 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:08.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:08.060 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T20:22:08.124 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:22:08.124 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:22:08.124 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:22:08.124 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:08.124 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-26T20:22:08.125 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:08.125 INFO:teuthology.orchestra.run.smithi037.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:22:08.125 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 96:0a:b1:4b:cb:2e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:22:08.125 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:22:08.125 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:08.125 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::940a:b1ff:fe4b:cb2e/64 scope link tentative 2024-04-26T20:22:08.125 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:08.126 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T20:22:08.126 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:08.126 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T20:22:08.126 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T20:22:08.127 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T20:22:08.127 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T20:22:08.127 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T20:22:08.127 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T20:22:08.127 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T20:22:08.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:08.280 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:08.280 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T20:22:08.280 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.1 up 2024-04-26T20:22:08.280 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T20:22:08.280 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T20:22:08.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:08.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:08.359 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:08.360 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T20:22:08.385 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T20:22:08.385 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:08.386 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T20:22:08.441 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-26T20:22:08.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:08.441 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:08.524 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:08.524 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:08.532 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:08.532 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:08.532 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-04-26T20:22:08.533 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with new device syntax: 2@d49b1c18-e1bf-49f2-acda-0904fa7c6684.cephfs=/ 2024-04-26T20:22:08.533 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.37:3300/172.21.15.134:3300/172.21.15.134:3301" will pass to kernel 2024-04-26T20:22:08.533 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-26T20:22:08.533 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:22:08.533 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:22:08.533 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:22:08.533 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:22:08.536 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:22:08.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:08.536 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T20:22:08.560 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:08.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:08.657 INFO:teuthology.orchestra.run.smithi037.stdout:client4499 2024-04-26T20:22:08.657 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:08.657 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:08.673 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:08.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:08.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:08.770 INFO:teuthology.orchestra.run.smithi037.stdout:client4499 2024-04-26T20:22:08.771 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:08.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:08.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:08.828 INFO:teuthology.orchestra.run.smithi037.stdout:client4499 2024-04-26T20:22:08.828 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d49b1c18-e1bf-49f2-acda-0904fa7c6684.client4499/status') 2024-04-26T20:22:08.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:08.887 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.4499 (3)172.21.15.37:0/3866125145 2024-04-26T20:22:08.888 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-04-26T20:22:08.888 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-04-26T20:22:08.888 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-04-26T20:22:08.888 INFO:teuthology.orchestra.run.smithi037.stderr:70 bytes copied, 2.3335e-05 s, 3.0 MB/s 2024-04-26T20:22:08.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:08.895 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-26T20:22:08.895 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:22:08.895 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:22:08.895 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:22:08.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.995+0000 7f2d7c904640 1 Processor -- start 2024-04-26T20:22:08.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.995+0000 7f2d7c904640 1 -- start start 2024-04-26T20:22:08.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.996+0000 7f2d7c904640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78073120 0x7f2d7806ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:08.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.996+0000 7f2d7c904640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d7815be00 con 0x7f2d780d0540 2024-04-26T20:22:08.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.996+0000 7f2d7c904640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d7815c0a0 con 0x7f2d7806f2c0 2024-04-26T20:22:08.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.996+0000 7f2d7c904640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7815c380 con 0x7f2d78073120 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2d780d0540 0x7f2d78072a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34496/0 (socket says 172.21.15.37:34496) 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7659c640 1 -- 172.21.15.37:0/277986602 learned_addr learned my addr 172.21.15.37:0/277986602 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d75d9b640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78073120 0x7f2d7806ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 805715768 0 0) 0x7f2d7815be00 con 0x7f2d780d0540 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d60003520 con 0x7f2d780d0540 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3778125672 0 0) 0x7f2d60003520 con 0x7f2d780d0540 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78073120 msgr2=0x7f2d7806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78073120 0x7f2d7806ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 >> v1:172.21.15.134:6789/0 conn(0x7f2d7806f2c0 legacy=0x7f2d780cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7815c660 con 0x7f2d780d0540 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f2d68001670 con 0x7f2d780d0540 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 >> v1:172.21.15.37:6789/0 conn(0x7f2d780d0540 legacy=0x7f2d78072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d600039e0 0x7f2d60003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:09.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 0x7f2d60004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78073120 0x7f2d600050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7815c380 con 0x7f2d60004390 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d600079a0 con 0x7f2d600039e0 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2d60007ba0 con 0x7f2d78073120 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d75d9b640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 0x7f2d60004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d7659c640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d600039e0 0x7f2d60003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.997+0000 7f2d76d9d640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78073120 0x7f2d600050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.998+0000 7f2d75d9b640 1 -- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78073120 msgr2=0x7f2d600050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.998+0000 7f2d75d9b640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78073120 0x7f2d600050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.998+0000 7f2d75d9b640 1 -- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d600039e0 msgr2=0x7f2d60003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.998+0000 7f2d75d9b640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d600039e0 0x7f2d60003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.998+0000 7f2d75d9b640 1 -- 172.21.15.37:0/277986602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d60003520 con 0x7f2d60004390 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.998+0000 7f2d76d9d640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78073120 0x7f2d600050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.998+0000 7f2d75d9b640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 0x7f2d60004860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009ac0 tx=0x7f2d60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18e745d38bd03cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:09.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.998+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c036070 con 0x7f2d60004390 2024-04-26T20:22:09.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.999+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c031400 con 0x7f2d60004390 2024-04-26T20:22:09.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.999+0000 7f2d7559a640 1 -- 172.21.15.37:0/277986602 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c009690 con 0x7f2d60004390 2024-04-26T20:22:09.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.999+0000 7f2d7c904640 1 -- 172.21.15.37:0/277986602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 msgr2=0x7f2d60004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:09.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.999+0000 7f2d7c904640 1 --2- 172.21.15.37:0/277986602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 0x7f2d60004860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009ac0 tx=0x7f2d60005830 comp rx=0 tx=0).stop 2024-04-26T20:22:09.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.999+0000 7f2d76d9d640 1 -- 172.21.15.37:0/277986602 reap_dead start 2024-04-26T20:22:09.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.999+0000 7f2d7c904640 1 -- 172.21.15.37:0/277986602 shutdown_connections 2024-04-26T20:22:09.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.999+0000 7f2d7c904640 1 -- 172.21.15.37:0/277986602 >> 172.21.15.37:0/277986602 conn(0x7f2d780d0d50 msgr2=0x7f2d780d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:09.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.999+0000 7f2d7c904640 1 -- 172.21.15.37:0/277986602 shutdown_connections 2024-04-26T20:22:09.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:08.999+0000 7f2d7c904640 1 -- 172.21.15.37:0/277986602 wait complete. 2024-04-26T20:22:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.001+0000 7f2d7c904640 1 Processor -- start 2024-04-26T20:22:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.001+0000 7f2d7c904640 1 -- start start 2024-04-26T20:22:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.001+0000 7f2d7c904640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 0x7f2d781f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.001+0000 7f2d7c904640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7814f4a0 0x7f2d781f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.001+0000 7f2d7c904640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78151b20 0x7f2d78151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.001+0000 7f2d7c904640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780d0aa0 con 0x7f2d60004390 2024-04-26T20:22:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.001+0000 7f2d7c904640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780d0920 con 0x7f2d7814f4a0 2024-04-26T20:22:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.001+0000 7f2d7c904640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7806ed80 con 0x7f2d78151b20 2024-04-26T20:22:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.001+0000 7f2d7659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 0x7f2d781f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.001+0000 7f2d7659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 0x7f2d781f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40110/0 (socket says 172.21.15.37:40110) 2024-04-26T20:22:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.001+0000 7f2d7659c640 1 -- 172.21.15.37:0/2559984887 learned_addr learned my addr 172.21.15.37:0/2559984887 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d75d9b640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7814f4a0 0x7f2d781f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d76d9d640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78151b20 0x7f2d78151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d7659c640 1 -- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78151b20 msgr2=0x7f2d78151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d7659c640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78151b20 0x7f2d78151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d7659c640 1 -- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7814f4a0 msgr2=0x7f2d781f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d7659c640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7814f4a0 0x7f2d781f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d7659c640 1 -- 172.21.15.37:0/2559984887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d781f4270 con 0x7f2d60004390 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d76d9d640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78151b20 0x7f2d78151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d7659c640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 0x7f2d781f3d30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2d68007290 tx=0x7f2d68007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d66ffd640 1 -- 172.21.15.37:0/2559984887 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800f960 con 0x7f2d60004390 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d66ffd640 1 -- 172.21.15.37:0/2559984887 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d68018070 con 0x7f2d60004390 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d7c904640 1 -- 172.21.15.37:0/2559984887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d781f5400 con 0x7f2d60004390 2024-04-26T20:22:09.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d66ffd640 1 -- 172.21.15.37:0/2559984887 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800fd60 con 0x7f2d60004390 2024-04-26T20:22:09.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.002+0000 7f2d7c904640 1 -- 172.21.15.37:0/2559984887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7814f960 con 0x7f2d60004390 2024-04-26T20:22:09.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.003+0000 7f2d7c904640 1 -- 172.21.15.37:0/2559984887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7806a360 con 0x7f2d60004390 2024-04-26T20:22:09.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.004+0000 7f2d66ffd640 1 -- 172.21.15.37:0/2559984887 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d6801d460 con 0x7f2d60004390 2024-04-26T20:22:09.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.004+0000 7f2d66ffd640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2d5806c920 0x7f2d5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:09.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.004+0000 7f2d66ffd640 1 -- 172.21.15.37:0/2559984887 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2d6808bba0 con 0x7f2d60004390 2024-04-26T20:22:09.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.004+0000 7f2d75d9b640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2d5806c920 0x7f2d5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:09.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.005+0000 7f2d75d9b640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2d5806c920 0x7f2d5806ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009ac0 tx=0x7f2d6c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:09.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.008+0000 7f2d66ffd640 1 -- 172.21.15.37:0/2559984887 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d781f5400 con 0x7f2d60004390 2024-04-26T20:22:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.235+0000 7f2d7c904640 1 -- 172.21.15.37:0/2559984887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2d7806f8e0 con 0x7f2d60004390 2024-04-26T20:22:09.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.236+0000 7f2d66ffd640 1 -- 172.21.15.37:0/2559984887 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f2d6805a060 con 0x7f2d60004390 2024-04-26T20:22:09.236 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:22:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 -- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2d5806c920 msgr2=0x7f2d5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2d5806c920 0x7f2d5806ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009ac0 tx=0x7f2d6c008030 comp rx=0 tx=0).stop 2024-04-26T20:22:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 -- 172.21.15.37:0/2559984887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 msgr2=0x7f2d781f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 0x7f2d781f3d30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2d68007290 tx=0x7f2d68007750 comp rx=0 tx=0).stop 2024-04-26T20:22:09.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 -- 172.21.15.37:0/2559984887 shutdown_connections 2024-04-26T20:22:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2d5806c920 0x7f2d5806ede0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d78151b20 0x7f2d78151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7814f4a0 0x7f2d781f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 --2- 172.21.15.37:0/2559984887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d60004390 0x7f2d781f3d30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 -- 172.21.15.37:0/2559984887 >> 172.21.15.37:0/2559984887 conn(0x7f2d780d0d50 msgr2=0x7f2d780cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 -- 172.21.15.37:0/2559984887 shutdown_connections 2024-04-26T20:22:09.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:09.238+0000 7f2d7c904640 1 -- 172.21.15.37:0/2559984887 wait complete. 2024-04-26T20:22:09.249 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:22:09.249 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:22:09.249 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi134.front.sepia.ceph.com 2024-04-26T20:22:09.249 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T20:22:09.249 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:22:09.249 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:22:09.249 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:22:09.249 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:22:09.249 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:22:09.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:09.249 DEBUG:teuthology.orchestra.run.smithi134:> ip addr 2024-04-26T20:22:09.268 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:22:09.268 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:22:09.268 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:22:09.268 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:09.268 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-04-26T20:22:09.268 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:09.268 INFO:teuthology.orchestra.run.smithi134.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:09.268 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e6 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:09.268 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f0 2024-04-26T20:22:09.269 INFO:teuthology.orchestra.run.smithi134.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:09.269 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e7 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:09.269 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f1 2024-04-26T20:22:09.269 INFO:teuthology.orchestra.run.smithi134.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:09.269 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e8 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:09.269 INFO:teuthology.orchestra.run.smithi134.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:22:09.269 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e9 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:09.269 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.21.15.134/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:22:09.269 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft 42745sec preferred_lft 42745sec 2024-04-26T20:22:09.269 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::5526:381c:36db:a6d8/64 scope link noprefixroute 2024-04-26T20:22:09.269 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:09.269 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:22:09.270 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:09.270 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-26T20:22:09.270 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:22:09.270 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip addr flush dev ceph-brx 2024-04-26T20:22:09.270 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set ceph-brx up 2024-04-26T20:22:09.270 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:22:09.270 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-26T20:22:09.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:09.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:09.461 DEBUG:teuthology.orchestra.run.smithi134:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:22:09.484 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-04-26T20:22:09.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:09.486 DEBUG:teuthology.orchestra.run.smithi134:> route 2024-04-26T20:22:09.548 INFO:teuthology.orchestra.run.smithi134.stdout:Kernel IP routing table 2024-04-26T20:22:09.548 INFO:teuthology.orchestra.run.smithi134.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:22:09.549 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:22:09.549 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:22:09.549 INFO:teuthology.orchestra.run.smithi134.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:22:09.549 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:09.549 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-26T20:22:09.549 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:22:09.549 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:22:09.549 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:22:09.549 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-26T20:22:09.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:09.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:09.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:09.832 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-26T20:22:09.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:09.847 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list-id 2024-04-26T20:22:09.902 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:09.902 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-26T20:22:09.902 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:22:09.902 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T20:22:09.902 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-26T20:22:09.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:10.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:10.023 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T20:22:10.023 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:10.023 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-26T20:22:10.023 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T20:22:10.023 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:22:10.023 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T20:22:10.023 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T20:22:10.023 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T20:22:10.024 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-26T20:22:10.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:10.143 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:10.143 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-26T20:22:10.143 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set brx.0 up 2024-04-26T20:22:10.143 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:22:10.143 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-26T20:22:10.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:10.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:10.221 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:10.221 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T20:22:10.235 INFO:teuthology.orchestra.run.smithi134.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:22:10.235 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:10.236 DEBUG:teuthology.orchestra.run.smithi134:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:22:10.290 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-26T20:22:10.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:10.290 DEBUG:teuthology.orchestra.run.smithi134:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:10.386 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:10.386 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:10.487 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:10.487 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:10.487 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-04-26T20:22:10.487 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: trying mount with new device syntax: 1@d49b1c18-e1bf-49f2-acda-0904fa7c6684.cephfs=/ 2024-04-26T20:22:10.487 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.37:3300/172.21.15.134:3300/172.21.15.134:3301" will pass to kernel 2024-04-26T20:22:10.488 INFO:teuthology.orchestra.run.smithi134.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T20:22:10.488 INFO:teuthology.orchestra.run.smithi134.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:22:10.488 INFO:teuthology.orchestra.run.smithi134.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:22:10.488 INFO:teuthology.orchestra.run.smithi134.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:22:10.488 INFO:teuthology.orchestra.run.smithi134.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:22:10.489 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:22:10.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:10.490 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:22:10.513 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:10.609 INFO:teuthology.orchestra.run.smithi134.stdout:client4618 2024-04-26T20:22:10.610 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:22:10.610 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:10.625 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:10.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:10.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:10.721 INFO:teuthology.orchestra.run.smithi134.stdout:client4618 2024-04-26T20:22:10.721 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:10.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:10.776 INFO:teuthology.orchestra.run.smithi134.stdout:client4618 2024-04-26T20:22:10.776 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d49b1c18-e1bf-49f2-acda-0904fa7c6684.client4618/status') 2024-04-26T20:22:10.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:10.833 INFO:teuthology.orchestra.run.smithi134.stdout:instance: client.4618 (3)192.168.0.1:0/1444852922 2024-04-26T20:22:10.833 INFO:teuthology.orchestra.run.smithi134.stdout:blocklisted: false 2024-04-26T20:22:10.833 INFO:teuthology.orchestra.run.smithi134.stderr:0+1 records in 2024-04-26T20:22:10.833 INFO:teuthology.orchestra.run.smithi134.stderr:0+1 records out 2024-04-26T20:22:10.833 INFO:teuthology.orchestra.run.smithi134.stderr:69 bytes copied, 2.256e-05 s, 3.1 MB/s 2024-04-26T20:22:10.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:10.840 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-26T20:22:10.840 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:22:10.840 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T20:22:10.840 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:22:10.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.953+0000 7f32d441d640 1 Processor -- start 2024-04-26T20:22:10.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.953+0000 7f32d441d640 1 -- start start 2024-04-26T20:22:10.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.954+0000 7f32d441d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f32cc0d3e50 0x7f32cc0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:10.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.954+0000 7f32d441d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32cc0d3220 con 0x7f32cc0d2180 2024-04-26T20:22:10.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.954+0000 7f32d441d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32cc0d5ef0 con 0x7f32cc15f400 2024-04-26T20:22:10.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.954+0000 7f32d441d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f32cc0d1290 con 0x7f32cc0d3e50 2024-04-26T20:22:10.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.955+0000 7f32d19b8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f32cc0d2180 0x7f32cc0d3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34500/0 (socket says 172.21.15.37:34500) 2024-04-26T20:22:10.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.955+0000 7f32d19b8640 1 -- 172.21.15.37:0/4291264310 learned_addr learned my addr 172.21.15.37:0/4291264310 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:10.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.955+0000 7f32d29ba640 1 --2- 172.21.15.37:0/4291264310 >> v2:172.21.15.134:3300/0 conn(0x7f32cc0d3e50 0x7f32cc0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:10.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.955+0000 7f32d11b7640 1 -- 172.21.15.37:0/4291264310 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3294009559 0 0) 0x7f32cc0d3220 con 0x7f32cc0d2180 2024-04-26T20:22:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.955+0000 7f32d11b7640 1 -- 172.21.15.37:0/4291264310 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32b4003520 con 0x7f32cc0d2180 2024-04-26T20:22:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.955+0000 7f32d29ba640 1 -- 172.21.15.37:0/4291264310 >> v1:172.21.15.134:6789/0 conn(0x7f32cc15f400 legacy=0x7f32cc0d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.955+0000 7f32d11b7640 1 -- 172.21.15.37:0/4291264310 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1357891672 0 0) 0x7f32cc0d5ef0 con 0x7f32cc15f400 2024-04-26T20:22:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.955+0000 7f32d29ba640 1 -- 172.21.15.37:0/4291264310 >> v1:172.21.15.37:6789/0 conn(0x7f32cc0d2180 legacy=0x7f32cc0d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.955+0000 7f32d29ba640 1 -- 172.21.15.37:0/4291264310 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc0d00a0 con 0x7f32cc0d3e50 2024-04-26T20:22:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.956+0000 7f32d29ba640 1 --2- 172.21.15.37:0/4291264310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc0d3e50 0x7f32cc0b1d70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009d90 tx=0x7f32bc02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e2cdad76cf44bde server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.956+0000 7f32d11b7640 1 -- 172.21.15.37:0/4291264310 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc007740 con 0x7f32cc0d3e50 2024-04-26T20:22:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.956+0000 7f32d11b7640 1 -- 172.21.15.37:0/4291264310 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32bc0078e0 con 0x7f32cc0d3e50 2024-04-26T20:22:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.956+0000 7f32d11b7640 1 -- 172.21.15.37:0/4291264310 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc007c90 con 0x7f32cc0d3e50 2024-04-26T20:22:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.957+0000 7f32d441d640 1 -- 172.21.15.37:0/4291264310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc0d3e50 msgr2=0x7f32cc0b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.957+0000 7f32d441d640 1 --2- 172.21.15.37:0/4291264310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc0d3e50 0x7f32cc0b1d70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009d90 tx=0x7f32bc02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:22:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.957+0000 7f32d441d640 1 -- 172.21.15.37:0/4291264310 shutdown_connections 2024-04-26T20:22:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.957+0000 7f32d441d640 1 --2- 172.21.15.37:0/4291264310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc0d3e50 0x7f32cc0b1d70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.957+0000 7f32d441d640 1 -- 172.21.15.37:0/4291264310 >> 172.21.15.37:0/4291264310 conn(0x7f32cc09e510 msgr2=0x7f32cc0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:10.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.957+0000 7f32d441d640 1 -- 172.21.15.37:0/4291264310 shutdown_connections 2024-04-26T20:22:10.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.957+0000 7f32d441d640 1 -- 172.21.15.37:0/4291264310 wait complete. 2024-04-26T20:22:10.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.958+0000 7f32d441d640 1 Processor -- start 2024-04-26T20:22:10.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.958+0000 7f32d441d640 1 -- start start 2024-04-26T20:22:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d441d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32cc0d2180 0x7f32cc162400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d441d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc0d3e50 0x7f32cc1f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d441d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32cc15f400 0x7f32cc0b91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d441d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc0d25f0 con 0x7f32cc15f400 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d441d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc0d0660 con 0x7f32cc0d3e50 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d441d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f32cc0cfd50 con 0x7f32cc0d2180 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d29ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32cc15f400 0x7f32cc0b91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d29ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32cc15f400 0x7f32cc0b91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40116/0 (socket says 172.21.15.37:40116) 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d29ba640 1 -- 172.21.15.37:0/1104521886 learned_addr learned my addr 172.21.15.37:0/1104521886 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d19b8640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc0d3e50 0x7f32cc1f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d21b9640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32cc0d2180 0x7f32cc162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d29ba640 1 -- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32cc0d2180 msgr2=0x7f32cc162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d29ba640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32cc0d2180 0x7f32cc162400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d29ba640 1 -- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc0d3e50 msgr2=0x7f32cc1f90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d29ba640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc0d3e50 0x7f32cc1f90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d29ba640 1 -- 172.21.15.37:0/1104521886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc1f9620 con 0x7f32cc15f400 2024-04-26T20:22:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.959+0000 7f32d29ba640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32cc15f400 0x7f32cc0b91d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009eb0 tx=0x7f32bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:10.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.960+0000 7f32caffd640 1 -- 172.21.15.37:0/1104521886 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc03f070 con 0x7f32cc15f400 2024-04-26T20:22:10.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.960+0000 7f32caffd640 1 -- 172.21.15.37:0/1104521886 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32bc031bc0 con 0x7f32cc15f400 2024-04-26T20:22:10.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.960+0000 7f32d441d640 1 -- 172.21.15.37:0/1104521886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc161360 con 0x7f32cc15f400 2024-04-26T20:22:10.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.960+0000 7f32caffd640 1 -- 172.21.15.37:0/1104521886 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc009450 con 0x7f32cc15f400 2024-04-26T20:22:10.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.960+0000 7f32d441d640 1 -- 172.21.15.37:0/1104521886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc161c00 con 0x7f32cc15f400 2024-04-26T20:22:10.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.961+0000 7f32d441d640 1 -- 172.21.15.37:0/1104521886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc0693e0 con 0x7f32cc15f400 2024-04-26T20:22:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.961+0000 7f32caffd640 1 -- 172.21.15.37:0/1104521886 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32bc043410 con 0x7f32cc15f400 2024-04-26T20:22:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.962+0000 7f32caffd640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f32b806c8f0 0x7f32b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.962+0000 7f32caffd640 1 -- 172.21.15.37:0/1104521886 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f32bc0acd80 con 0x7f32cc15f400 2024-04-26T20:22:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.962+0000 7f32d21b9640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f32b806c8f0 0x7f32b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.962+0000 7f32d21b9640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f32b806c8f0 0x7f32b806edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f32c4000f60 tx=0x7f32c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:10.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:10.966+0000 7f32caffd640 1 -- 172.21.15.37:0/1104521886 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f32bc07abf0 con 0x7f32cc15f400 2024-04-26T20:22:11.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.192+0000 7f32d441d640 1 -- 172.21.15.37:0/1104521886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f32cc0d2880 con 0x7f32cc15f400 2024-04-26T20:22:11.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.192+0000 7f32caffd640 1 -- 172.21.15.37:0/1104521886 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f32bc07eef0 con 0x7f32cc15f400 2024-04-26T20:22:11.193 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:22:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 -- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f32b806c8f0 msgr2=0x7f32b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f32b806c8f0 0x7f32b806edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f32c4000f60 tx=0x7f32c4009040 comp rx=0 tx=0).stop 2024-04-26T20:22:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 -- 172.21.15.37:0/1104521886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32cc15f400 msgr2=0x7f32cc0b91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32cc15f400 0x7f32cc0b91d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009eb0 tx=0x7f32bc004a40 comp rx=0 tx=0).stop 2024-04-26T20:22:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 -- 172.21.15.37:0/1104521886 shutdown_connections 2024-04-26T20:22:11.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32cc15f400 0x7f32cc0b91d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:11.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc0d3e50 0x7f32cc1f90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:11.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f32b806c8f0 0x7f32b806edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:11.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 --2- 172.21.15.37:0/1104521886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32cc0d2180 0x7f32cc162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:11.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 -- 172.21.15.37:0/1104521886 >> 172.21.15.37:0/1104521886 conn(0x7f32cc09e510 msgr2=0x7f32cc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:11.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 -- 172.21.15.37:0/1104521886 shutdown_connections 2024-04-26T20:22:11.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:11.194+0000 7f32d441d640 1 -- 172.21.15.37:0/1104521886 wait complete. 2024-04-26T20:22:11.206 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:22:11.206 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:22:11.206 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi134.front.sepia.ceph.com 2024-04-26T20:22:11.206 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T20:22:11.206 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T20:22:11.206 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:22:11.206 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:22:11.206 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:22:11.206 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T20:22:11.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:11.206 DEBUG:teuthology.orchestra.run.smithi134:> ip addr 2024-04-26T20:22:11.222 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:22:11.222 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:22:11.222 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:22:11.222 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e6 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f0 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e7 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f1 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e8 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e9 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.21.15.134/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft 42743sec preferred_lft 42743sec 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::5526:381c:36db:a6d8/64 scope link noprefixroute 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether c2:13:60:38:f8:42 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::205c:9aff:fee0:b7b/64 scope link tentative 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether c2:13:60:38:f8:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::c013:60ff:fe38:f842/64 scope link tentative 2024-04-26T20:22:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:11.225 DEBUG:teuthology.orchestra.run.smithi134:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:22:11.285 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-04-26T20:22:11.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:11.287 DEBUG:teuthology.orchestra.run.smithi134:> route 2024-04-26T20:22:11.343 INFO:teuthology.orchestra.run.smithi134.stdout:Kernel IP routing table 2024-04-26T20:22:11.344 INFO:teuthology.orchestra.run.smithi134.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:22:11.344 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:22:11.344 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:22:11.344 INFO:teuthology.orchestra.run.smithi134.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:22:11.344 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:11.344 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-26T20:22:11.344 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:22:11.344 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:22:11.344 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:22:11.345 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-26T20:22:11.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:11.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:11.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:11.467 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-26T20:22:11.481 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:22:11.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:11.482 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list-id 2024-04-26T20:22:11.536 INFO:teuthology.orchestra.run.smithi134.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T20:22:11.536 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:11.536 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-26T20:22:11.536 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:22:11.536 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T20:22:11.536 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-26T20:22:11.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:11.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:11.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:11.652 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 42:ed:c3:57:b2:90 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::40ed:c3ff:fe57:b290/64 scope link 2024-04-26T20:22:11.675 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:11.676 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T20:22:11.739 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:22:11.739 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:22:11.740 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:22:11.740 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.740 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-04-26T20:22:11.740 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.740 INFO:teuthology.orchestra.run.smithi134.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:22:11.740 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 42:ed:c3:57:b2:90 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:22:11.740 INFO:teuthology.orchestra.run.smithi134.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:22:11.740 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.740 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::40ed:c3ff:fe57:b290/64 scope link 2024-04-26T20:22:11.740 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:11.741 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T20:22:11.741 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:11.741 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-26T20:22:11.742 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T20:22:11.742 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T20:22:11.742 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T20:22:11.742 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T20:22:11.742 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T20:22:11.742 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-26T20:22:11.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:11.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:11.889 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:11.889 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-26T20:22:11.889 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set brx.1 up 2024-04-26T20:22:11.889 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T20:22:11.889 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-26T20:22:11.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:11.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:11.963 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:11.963 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T20:22:11.978 INFO:teuthology.orchestra.run.smithi134.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T20:22:11.978 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:11.978 DEBUG:teuthology.orchestra.run.smithi134:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T20:22:12.033 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-26T20:22:12.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:12.033 DEBUG:teuthology.orchestra.run.smithi134:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:12.114 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:12.114 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:12.122 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:12.122 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:12.122 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-04-26T20:22:12.122 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: trying mount with new device syntax: 3@d49b1c18-e1bf-49f2-acda-0904fa7c6684.cephfs=/ 2024-04-26T20:22:12.122 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.37:3300/172.21.15.134:3300/172.21.15.134:3301" will pass to kernel 2024-04-26T20:22:12.124 INFO:teuthology.orchestra.run.smithi134.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-26T20:22:12.124 INFO:teuthology.orchestra.run.smithi134.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:22:12.125 INFO:teuthology.orchestra.run.smithi134.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:22:12.125 INFO:teuthology.orchestra.run.smithi134.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:22:12.125 INFO:teuthology.orchestra.run.smithi134.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:22:12.125 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:22:12.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:12.125 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T20:22:12.196 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:12.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:12.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:12.291 INFO:teuthology.orchestra.run.smithi134.stdout:client4630 2024-04-26T20:22:12.291 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:22:12.292 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:12.307 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:12.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:12.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:12.401 INFO:teuthology.orchestra.run.smithi134.stdout:client4630 2024-04-26T20:22:12.401 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:12.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:12.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:12.456 INFO:teuthology.orchestra.run.smithi134.stdout:client4630 2024-04-26T20:22:12.456 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d49b1c18-e1bf-49f2-acda-0904fa7c6684.client4630/status') 2024-04-26T20:22:12.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:12.513 INFO:teuthology.orchestra.run.smithi134.stdout:instance: client.4630 (3)192.168.0.2:0/2606920813 2024-04-26T20:22:12.513 INFO:teuthology.orchestra.run.smithi134.stdout:blocklisted: false 2024-04-26T20:22:12.513 INFO:teuthology.orchestra.run.smithi134.stderr:0+1 records in 2024-04-26T20:22:12.513 INFO:teuthology.orchestra.run.smithi134.stderr:0+1 records out 2024-04-26T20:22:12.513 INFO:teuthology.orchestra.run.smithi134.stderr:69 bytes copied, 3.8339e-05 s, 1.8 MB/s 2024-04-26T20:22:12.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:12.520 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T20:22:12.536 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:12.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.649+0000 7f274dcfc640 1 Processor -- start 2024-04-26T20:22:12.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.649+0000 7f274dcfc640 1 -- start start 2024-04-26T20:22:12.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.650+0000 7f274dcfc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2748162e40 0x7f27480d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.650+0000 7f274dcfc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f27480b2050 0x7f27480cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.650+0000 7f274dcfc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27480d0150 con 0x7f27480cfd70 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.650+0000 7f274dcfc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2748163220 con 0x7f27480b2050 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.650+0000 7f274dcfc640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f27480d0eb0 con 0x7f2748162e40 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.650+0000 7f2746ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f27480b2050 0x7f27480cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.650+0000 7f2747fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f27480cfd70 0x7f27480ce010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34508/0 (socket says 172.21.15.37:34508) 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.650+0000 7f2747fff640 1 -- 172.21.15.37:0/3862730521 learned_addr learned my addr 172.21.15.37:0/3862730521 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.650+0000 7f27477fe640 1 --2- 172.21.15.37:0/3862730521 >> v2:172.21.15.134:3300/0 conn(0x7f2748162e40 0x7f27480d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.651+0000 7f2746ffd640 1 -- 172.21.15.37:0/3862730521 >> v2:172.21.15.134:3300/0 conn(0x7f2748162e40 msgr2=0x7f27480d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.651+0000 7f2746ffd640 1 --2- 172.21.15.37:0/3862730521 >> v2:172.21.15.134:3300/0 conn(0x7f2748162e40 0x7f27480d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.651+0000 7f2746ffd640 1 -- 172.21.15.37:0/3862730521 >> v1:172.21.15.37:6789/0 conn(0x7f27480cfd70 legacy=0x7f27480ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.651+0000 7f2746ffd640 1 -- 172.21.15.37:0/3862730521 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27480b1d70 con 0x7f27480b2050 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.651+0000 7f27467fc640 1 -- 172.21.15.37:0/3862730521 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1920150042 0 0) 0x7f27480d0150 con 0x7f27480cfd70 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.651+0000 7f27477fe640 1 --2- 172.21.15.37:0/3862730521 >> v2:172.21.15.134:3300/0 conn(0x7f2748162e40 0x7f27480d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:12.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.651+0000 7f2746ffd640 1 --2- 172.21.15.37:0/3862730521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27480b2050 0x7f27480cf830 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2748075e20 tx=0x7f273c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5bc7764999bae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:12.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.651+0000 7f27467fc640 1 -- 172.21.15.37:0/3862730521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c037070 con 0x7f27480b2050 2024-04-26T20:22:12.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.651+0000 7f27467fc640 1 -- 172.21.15.37:0/3862730521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c032440 con 0x7f27480b2050 2024-04-26T20:22:12.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.651+0000 7f27467fc640 1 -- 172.21.15.37:0/3862730521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c02ae60 con 0x7f27480b2050 2024-04-26T20:22:12.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.652+0000 7f274dcfc640 1 -- 172.21.15.37:0/3862730521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27480b2050 msgr2=0x7f27480cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:12.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.652+0000 7f274dcfc640 1 --2- 172.21.15.37:0/3862730521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27480b2050 0x7f27480cf830 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2748075e20 tx=0x7f273c02a110 comp rx=0 tx=0).stop 2024-04-26T20:22:12.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.652+0000 7f274dcfc640 1 -- 172.21.15.37:0/3862730521 shutdown_connections 2024-04-26T20:22:12.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.652+0000 7f274dcfc640 1 --2- 172.21.15.37:0/3862730521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27480b2050 0x7f27480cf830 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.652+0000 7f274dcfc640 1 --2- 172.21.15.37:0/3862730521 >> v2:172.21.15.134:3300/0 conn(0x7f2748162e40 0x7f27480d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.652+0000 7f274dcfc640 1 -- 172.21.15.37:0/3862730521 >> 172.21.15.37:0/3862730521 conn(0x7f274809ba50 msgr2=0x7f274809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.652+0000 7f274dcfc640 1 -- 172.21.15.37:0/3862730521 shutdown_connections 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.652+0000 7f274dcfc640 1 -- 172.21.15.37:0/3862730521 wait complete. 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.653+0000 7f274dcfc640 1 Processor -- start 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f274dcfc640 1 -- start start 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f274dcfc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27480b2050 0x7f274815fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f274dcfc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27480cfd70 0x7f274815a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f274dcfc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2748162e40 0x7f27481f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f274dcfc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27480d1f20 con 0x7f2748162e40 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f274dcfc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27480d1c40 con 0x7f27480cfd70 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f274dcfc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27480b24e0 con 0x7f27480b2050 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f2747fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2748162e40 0x7f27481f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f2747fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2748162e40 0x7f27481f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40130/0 (socket says 172.21.15.37:40130) 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f2747fff640 1 -- 172.21.15.37:0/1729861852 learned_addr learned my addr 172.21.15.37:0/1729861852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f2746ffd640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27480cfd70 0x7f274815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.654+0000 7f27477fe640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27480b2050 0x7f274815fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.655+0000 7f2747fff640 1 -- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27480b2050 msgr2=0x7f274815fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.655+0000 7f2747fff640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27480b2050 0x7f274815fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.655+0000 7f2747fff640 1 -- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27480cfd70 msgr2=0x7f274815a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.655+0000 7f2747fff640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27480cfd70 0x7f274815a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.655+0000 7f2747fff640 1 -- 172.21.15.37:0/1729861852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27481f5830 con 0x7f2748162e40 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.655+0000 7f2747fff640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2748162e40 0x7f27481f52f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f274000caf0 tx=0x7f274000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.655+0000 7f272ffff640 1 -- 172.21.15.37:0/1729861852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274000aca0 con 0x7f2748162e40 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.655+0000 7f272ffff640 1 -- 172.21.15.37:0/1729861852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f274000ae40 con 0x7f2748162e40 2024-04-26T20:22:12.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.655+0000 7f274dcfc640 1 -- 172.21.15.37:0/1729861852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27481f5ab0 con 0x7f2748162e40 2024-04-26T20:22:12.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.655+0000 7f272ffff640 1 -- 172.21.15.37:0/1729861852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740012620 con 0x7f2748162e40 2024-04-26T20:22:12.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.655+0000 7f274dcfc640 1 -- 172.21.15.37:0/1729861852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27481f6070 con 0x7f2748162e40 2024-04-26T20:22:12.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.656+0000 7f274dcfc640 1 -- 172.21.15.37:0/1729861852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274806a360 con 0x7f2748162e40 2024-04-26T20:22:12.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.656+0000 7f272ffff640 1 -- 172.21.15.37:0/1729861852 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2740012880 con 0x7f2748162e40 2024-04-26T20:22:12.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.657+0000 7f272ffff640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273406c8f0 0x7f273406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.657+0000 7f272ffff640 1 -- 172.21.15.37:0/1729861852 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f274008c000 con 0x7f2748162e40 2024-04-26T20:22:12.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.657+0000 7f27477fe640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273406c8f0 0x7f273406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.658+0000 7f27477fe640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273406c8f0 0x7f273406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2738006fd0 tx=0x7f2738008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:12.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.661+0000 7f272ffff640 1 -- 172.21.15.37:0/1729861852 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f27481f5830 con 0x7f2748162e40 2024-04-26T20:22:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.847+0000 7f274dcfc640 1 -- 172.21.15.37:0/1729861852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27481f62b0 con 0x7f2748162e40 2024-04-26T20:22:12.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.848+0000 7f272ffff640 1 -- 172.21.15.37:0/1729861852 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f2740059e70 con 0x7f2748162e40 2024-04-26T20:22:12.849 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:12.849 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 -- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273406c8f0 msgr2=0x7f273406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273406c8f0 0x7f273406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2738006fd0 tx=0x7f2738008040 comp rx=0 tx=0).stop 2024-04-26T20:22:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 -- 172.21.15.37:0/1729861852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2748162e40 msgr2=0x7f27481f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2748162e40 0x7f27481f52f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f274000caf0 tx=0x7f274000a060 comp rx=0 tx=0).stop 2024-04-26T20:22:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 -- 172.21.15.37:0/1729861852 shutdown_connections 2024-04-26T20:22:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2748162e40 0x7f27481f52f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27480cfd70 0x7f274815a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273406c8f0 0x7f273406edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 --2- 172.21.15.37:0/1729861852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27480b2050 0x7f274815fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 -- 172.21.15.37:0/1729861852 >> 172.21.15.37:0/1729861852 conn(0x7f274809ba50 msgr2=0x7f274815c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:12.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 -- 172.21.15.37:0/1729861852 shutdown_connections 2024-04-26T20:22:12.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.850+0000 7f274dcfc640 1 -- 172.21.15.37:0/1729861852 wait complete. 2024-04-26T20:22:12.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:12.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:12.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.966+0000 7f85bb459640 1 Processor -- start 2024-04-26T20:22:12.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.966+0000 7f85bb459640 1 -- start start 2024-04-26T20:22:12.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85bb459640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f85b415f790 0x7f85b40ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85bb459640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f85b40cb890 0x7f85b40ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85bb459640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 0x7f85b40b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85bb459640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f85b40b2c20 con 0x7f85b40cb890 2024-04-26T20:22:12.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85bb459640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f85b40cd430 con 0x7f85b415f790 2024-04-26T20:22:12.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85bb459640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f85b40cbe50 con 0x7f85b40b2250 2024-04-26T20:22:12.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b89f4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f85b40cb890 0x7f85b40ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b89f4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f85b40cb890 0x7f85b40ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40136/0 (socket says 172.21.15.37:40136) 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b89f4640 1 -- 172.21.15.37:0/1460759656 learned_addr learned my addr 172.21.15.37:0/1460759656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b99f6640 1 --2- 172.21.15.37:0/1460759656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 0x7f85b40b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b91f5640 1 --2- 172.21.15.37:0/1460759656 >> v2:172.21.15.134:3300/0 conn(0x7f85b415f790 0x7f85b40ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b89f4640 1 -- 172.21.15.37:0/1460759656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 msgr2=0x7f85b40b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b89f4640 1 --2- 172.21.15.37:0/1460759656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 0x7f85b40b26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b89f4640 1 -- 172.21.15.37:0/1460759656 >> v2:172.21.15.134:3300/0 conn(0x7f85b415f790 msgr2=0x7f85b40ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b89f4640 1 --2- 172.21.15.37:0/1460759656 >> v2:172.21.15.134:3300/0 conn(0x7f85b415f790 0x7f85b40ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b89f4640 1 -- 172.21.15.37:0/1460759656 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b40cc130 con 0x7f85b40cb890 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b99f6640 1 --2- 172.21.15.37:0/1460759656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 0x7f85b40b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.967+0000 7f85b91f5640 1 --2- 172.21.15.37:0/1460759656 >> v2:172.21.15.134:3300/0 conn(0x7f85b415f790 0x7f85b40ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.968+0000 7f85b89f4640 1 --2- 172.21.15.37:0/1460759656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85b40cb890 0x7f85b40ccef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f85b40d22e0 tx=0x7f85a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce41336e7c2dcd7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:12.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.968+0000 7f85abfff640 1 -- 172.21.15.37:0/1460759656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4037070 con 0x7f85b40cb890 2024-04-26T20:22:12.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.968+0000 7f85abfff640 1 -- 172.21.15.37:0/1460759656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a4032440 con 0x7f85b40cb890 2024-04-26T20:22:12.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.968+0000 7f85abfff640 1 -- 172.21.15.37:0/1460759656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4032810 con 0x7f85b40cb890 2024-04-26T20:22:12.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.969+0000 7f85bb459640 1 -- 172.21.15.37:0/1460759656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85b40cb890 msgr2=0x7f85b40ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:12.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.969+0000 7f85bb459640 1 --2- 172.21.15.37:0/1460759656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85b40cb890 0x7f85b40ccef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f85b40d22e0 tx=0x7f85a402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.969+0000 7f85bb459640 1 -- 172.21.15.37:0/1460759656 shutdown_connections 2024-04-26T20:22:12.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.969+0000 7f85bb459640 1 --2- 172.21.15.37:0/1460759656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 0x7f85b40b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.969+0000 7f85bb459640 1 --2- 172.21.15.37:0/1460759656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85b40cb890 0x7f85b40ccef0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.969+0000 7f85bb459640 1 --2- 172.21.15.37:0/1460759656 >> v2:172.21.15.134:3300/0 conn(0x7f85b415f790 0x7f85b40ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.969+0000 7f85bb459640 1 -- 172.21.15.37:0/1460759656 >> 172.21.15.37:0/1460759656 conn(0x7f85b40737f0 msgr2=0x7f85b4073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:12.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.969+0000 7f85bb459640 1 -- 172.21.15.37:0/1460759656 shutdown_connections 2024-04-26T20:22:12.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.969+0000 7f85bb459640 1 -- 172.21.15.37:0/1460759656 wait complete. 2024-04-26T20:22:12.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.970+0000 7f85bb459640 1 Processor -- start 2024-04-26T20:22:12.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.970+0000 7f85bb459640 1 -- start start 2024-04-26T20:22:12.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85bb459640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 0x7f85b409ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85bb459640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85b40cb890 0x7f85b40ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85bb459640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b415f790 0x7f85b40ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85bb459640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f85b40cc9d0 con 0x7f85b40cb890 2024-04-26T20:22:12.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85bb459640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85b40cc6f0 con 0x7f85b415f790 2024-04-26T20:22:12.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85bb459640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f85b40d0ba0 con 0x7f85b40b2250 2024-04-26T20:22:12.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b89f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85b40cb890 0x7f85b40ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b89f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85b40cb890 0x7f85b40ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40142/0 (socket says 172.21.15.37:40142) 2024-04-26T20:22:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b89f4640 1 -- 172.21.15.37:0/4268669145 learned_addr learned my addr 172.21.15.37:0/4268669145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b91f5640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 0x7f85b409ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b99f6640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b415f790 0x7f85b40ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b89f4640 1 -- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 msgr2=0x7f85b409ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b89f4640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 0x7f85b409ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b89f4640 1 -- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b415f790 msgr2=0x7f85b40ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b89f4640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b415f790 0x7f85b40ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b89f4640 1 -- 172.21.15.37:0/4268669145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b40c6830 con 0x7f85b40cb890 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b91f5640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 0x7f85b409ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b99f6640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b415f790 0x7f85b40ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85b89f4640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85b40cb890 0x7f85b40ae040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f85a402a220 tx=0x7f85a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85a9ffb640 1 -- 172.21.15.37:0/4268669145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4037040 con 0x7f85b40cb890 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.971+0000 7f85a9ffb640 1 -- 172.21.15.37:0/4268669145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a4042070 con 0x7f85b40cb890 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.972+0000 7f85bb459640 1 -- 172.21.15.37:0/4268669145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b40c4ea0 con 0x7f85b40cb890 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.972+0000 7f85a9ffb640 1 -- 172.21.15.37:0/4268669145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4030bb0 con 0x7f85b40cb890 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.972+0000 7f85bb459640 1 -- 172.21.15.37:0/4268669145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b40c5420 con 0x7f85b40cb890 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.973+0000 7f85bb459640 1 -- 172.21.15.37:0/4268669145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b406a360 con 0x7f85b40cb890 2024-04-26T20:22:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.973+0000 7f85a9ffb640 1 -- 172.21.15.37:0/4268669145 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85a4047460 con 0x7f85b40cb890 2024-04-26T20:22:12.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.973+0000 7f85a9ffb640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f859806c8f0 0x7f859806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:12.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.974+0000 7f85a9ffb640 1 -- 172.21.15.37:0/4268669145 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f85a40ac710 con 0x7f85b40cb890 2024-04-26T20:22:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.974+0000 7f85b91f5640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f859806c8f0 0x7f859806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.974+0000 7f85b91f5640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f859806c8f0 0x7f859806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f85b000aa30 tx=0x7f85b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:12.978+0000 7f85a9ffb640 1 -- 172.21.15.37:0/4268669145 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85b40c6830 con 0x7f85b40cb890 2024-04-26T20:22:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.161+0000 7f85bb459640 1 -- 172.21.15.37:0/4268669145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85b406d3e0 con 0x7f85b40cb890 2024-04-26T20:22:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.161+0000 7f85a9ffb640 1 -- 172.21.15.37:0/4268669145 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f85a4045360 con 0x7f85b40cb890 2024-04-26T20:22:13.163 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:13.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 -- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f859806c8f0 msgr2=0x7f859806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f859806c8f0 0x7f859806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f85b000aa30 tx=0x7f85b0008040 comp rx=0 tx=0).stop 2024-04-26T20:22:13.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 -- 172.21.15.37:0/4268669145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85b40cb890 msgr2=0x7f85b40ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85b40cb890 0x7f85b40ae040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f85a402a220 tx=0x7f85a40046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 -- 172.21.15.37:0/4268669145 shutdown_connections 2024-04-26T20:22:13.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b415f790 0x7f85b40ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85b40cb890 0x7f85b40ae040 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f859806c8f0 0x7f859806edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 --2- 172.21.15.37:0/4268669145 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85b40b2250 0x7f85b409ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 -- 172.21.15.37:0/4268669145 >> 172.21.15.37:0/4268669145 conn(0x7f85b40737f0 msgr2=0x7f85b4073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:13.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 -- 172.21.15.37:0/4268669145 shutdown_connections 2024-04-26T20:22:13.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.164+0000 7f85bb459640 1 -- 172.21.15.37:0/4268669145 wait complete. 2024-04-26T20:22:13.176 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.278+0000 7ffbef099640 1 Processor -- start 2024-04-26T20:22:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.278+0000 7ffbef099640 1 -- start start 2024-04-26T20:22:13.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbef099640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffbe815f9c0 0x7ffbe80d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbef099640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffbe80cf990 0x7ffbe80cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbef099640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbe8163380 0x7ffbe8163810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbef099640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ffbe80d0340 con 0x7ffbe815f9c0 2024-04-26T20:22:13.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbef099640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ffbe80b2550 con 0x7ffbe80cf990 2024-04-26T20:22:13.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbef099640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe806dcf0 con 0x7ffbe8163380 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbece35640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffbe815f9c0 0x7ffbe80d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbece35640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffbe815f9c0 0x7ffbe80d1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40156/0 (socket says 172.21.15.37:40156) 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbece35640 1 -- 172.21.15.37:0/728928030 learned_addr learned my addr 172.21.15.37:0/728928030 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbe7fff640 1 --2- 172.21.15.37:0/728928030 >> v2:172.21.15.134:3300/0 conn(0x7ffbe80cf990 0x7ffbe80cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbed636640 1 --2- 172.21.15.37:0/728928030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbe8163380 0x7ffbe8163810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbece35640 1 -- 172.21.15.37:0/728928030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbe8163380 msgr2=0x7ffbe8163810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbece35640 1 --2- 172.21.15.37:0/728928030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbe8163380 0x7ffbe8163810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbece35640 1 -- 172.21.15.37:0/728928030 >> v2:172.21.15.134:3300/0 conn(0x7ffbe80cf990 msgr2=0x7ffbe80cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbece35640 1 --2- 172.21.15.37:0/728928030 >> v2:172.21.15.134:3300/0 conn(0x7ffbe80cf990 0x7ffbe80cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.279+0000 7ffbece35640 1 -- 172.21.15.37:0/728928030 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe806dfd0 con 0x7ffbe815f9c0 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.280+0000 7ffbece35640 1 --2- 172.21.15.37:0/728928030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbe815f9c0 0x7ffbe80d1860 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80735b0 tx=0x7ffbe002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e024050f3bc138 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.280+0000 7ffbe77fe640 1 -- 172.21.15.37:0/728928030 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0037070 con 0x7ffbe815f9c0 2024-04-26T20:22:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.280+0000 7ffbe77fe640 1 -- 172.21.15.37:0/728928030 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbe0032440 con 0x7ffbe815f9c0 2024-04-26T20:22:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.280+0000 7ffbe77fe640 1 -- 172.21.15.37:0/728928030 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0032810 con 0x7ffbe815f9c0 2024-04-26T20:22:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.281+0000 7ffbef099640 1 -- 172.21.15.37:0/728928030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbe815f9c0 msgr2=0x7ffbe80d1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.281+0000 7ffbef099640 1 --2- 172.21.15.37:0/728928030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbe815f9c0 0x7ffbe80d1860 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80735b0 tx=0x7ffbe002a140 comp rx=0 tx=0).stop 2024-04-26T20:22:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.281+0000 7ffbef099640 1 -- 172.21.15.37:0/728928030 shutdown_connections 2024-04-26T20:22:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.281+0000 7ffbef099640 1 --2- 172.21.15.37:0/728928030 >> v2:172.21.15.134:3300/0 conn(0x7ffbe80cf990 0x7ffbe80cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.281+0000 7ffbef099640 1 --2- 172.21.15.37:0/728928030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbe8163380 0x7ffbe8163810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.281+0000 7ffbef099640 1 --2- 172.21.15.37:0/728928030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbe815f9c0 0x7ffbe80d1860 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.281+0000 7ffbef099640 1 -- 172.21.15.37:0/728928030 >> 172.21.15.37:0/728928030 conn(0x7ffbe80d2f80 msgr2=0x7ffbe80d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.281+0000 7ffbef099640 1 -- 172.21.15.37:0/728928030 shutdown_connections 2024-04-26T20:22:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.281+0000 7ffbef099640 1 -- 172.21.15.37:0/728928030 wait complete. 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.282+0000 7ffbef099640 1 Processor -- start 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.282+0000 7ffbef099640 1 -- start start 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbef099640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbe80cf990 0x7ffbe815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbef099640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbe815f9c0 0x7ffbe814d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbef099640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbe8163380 0x7ffbe8161e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbef099640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe806e870 con 0x7ffbe8163380 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbef099640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe806e590 con 0x7ffbe815f9c0 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbef099640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe806d910 con 0x7ffbe80cf990 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbed636640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbe8163380 0x7ffbe8161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbed636640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbe8163380 0x7ffbe8161e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40170/0 (socket says 172.21.15.37:40170) 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbed636640 1 -- 172.21.15.37:0/305526067 learned_addr learned my addr 172.21.15.37:0/305526067 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbe7fff640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbe815f9c0 0x7ffbe814d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbece35640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbe80cf990 0x7ffbe815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbed636640 1 -- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbe80cf990 msgr2=0x7ffbe815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbed636640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbe80cf990 0x7ffbe815ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbed636640 1 -- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbe815f9c0 msgr2=0x7ffbe814d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbed636640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbe815f9c0 0x7ffbe814d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.283+0000 7ffbed636640 1 -- 172.21.15.37:0/305526067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe8157800 con 0x7ffbe8163380 2024-04-26T20:22:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.284+0000 7ffbed636640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbe8163380 0x7ffbe8161e90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000d800 tx=0x7ffbd000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.284+0000 7ffbe57fa640 1 -- 172.21.15.37:0/305526067 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd000b660 con 0x7ffbe8163380 2024-04-26T20:22:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.284+0000 7ffbe57fa640 1 -- 172.21.15.37:0/305526067 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd0019070 con 0x7ffbe8163380 2024-04-26T20:22:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.284+0000 7ffbef099640 1 -- 172.21.15.37:0/305526067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe8160df0 con 0x7ffbe8163380 2024-04-26T20:22:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.284+0000 7ffbe57fa640 1 -- 172.21.15.37:0/305526067 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd000ba60 con 0x7ffbe8163380 2024-04-26T20:22:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.284+0000 7ffbef099640 1 -- 172.21.15.37:0/305526067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe81613b0 con 0x7ffbe8163380 2024-04-26T20:22:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.285+0000 7ffbef099640 1 -- 172.21.15.37:0/305526067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe806a360 con 0x7ffbe8163380 2024-04-26T20:22:13.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.285+0000 7ffbe57fa640 1 -- 172.21.15.37:0/305526067 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffbd000bc00 con 0x7ffbe8163380 2024-04-26T20:22:13.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.286+0000 7ffbe57fa640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffbc806c8f0 0x7ffbc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.286+0000 7ffbe57fa640 1 -- 172.21.15.37:0/305526067 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffbd008be40 con 0x7ffbe8163380 2024-04-26T20:22:13.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.286+0000 7ffbece35640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffbc806c8f0 0x7ffbc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.287+0000 7ffbece35640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffbc806c8f0 0x7ffbc806edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0034000 tx=0x7ffbe002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.290+0000 7ffbe57fa640 1 -- 172.21.15.37:0/305526067 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffbd0059c20 con 0x7ffbe8163380 2024-04-26T20:22:13.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.473+0000 7ffbef099640 1 -- 172.21.15.37:0/305526067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffbe8070df0 con 0x7ffbe8163380 2024-04-26T20:22:13.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.473+0000 7ffbe57fa640 1 -- 172.21.15.37:0/305526067 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ffbd005df20 con 0x7ffbe8163380 2024-04-26T20:22:13.475 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:13.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 -- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffbc806c8f0 msgr2=0x7ffbc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffbc806c8f0 0x7ffbc806edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0034000 tx=0x7ffbe002e000 comp rx=0 tx=0).stop 2024-04-26T20:22:13.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 -- 172.21.15.37:0/305526067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbe8163380 msgr2=0x7ffbe8161e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbe8163380 0x7ffbe8161e90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000d800 tx=0x7ffbd000dcc0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 -- 172.21.15.37:0/305526067 shutdown_connections 2024-04-26T20:22:13.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbe8163380 0x7ffbe8161e90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbe815f9c0 0x7ffbe814d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffbc806c8f0 0x7ffbc806edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 --2- 172.21.15.37:0/305526067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbe80cf990 0x7ffbe815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 -- 172.21.15.37:0/305526067 >> 172.21.15.37:0/305526067 conn(0x7ffbe80d2f80 msgr2=0x7ffbe80b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:13.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 -- 172.21.15.37:0/305526067 shutdown_connections 2024-04-26T20:22:13.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.475+0000 7ffbef099640 1 -- 172.21.15.37:0/305526067 wait complete. 2024-04-26T20:22:13.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.590+0000 7f343fd6a640 1 Processor -- start 2024-04-26T20:22:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.591+0000 7f343fd6a640 1 -- start start 2024-04-26T20:22:13.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.591+0000 7f343fd6a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f343815f3c0 0x7f343806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.591+0000 7f343fd6a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f34380d1990 0x7f343806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.591+0000 7f343fd6a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3438074050 con 0x7f34380cfa50 2024-04-26T20:22:13.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.591+0000 7f343fd6a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f34380b1060 con 0x7f343815f3c0 2024-04-26T20:22:13.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.591+0000 7f343fd6a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f34380b1340 con 0x7f34380d1990 2024-04-26T20:22:13.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.592+0000 7f343db06640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f343815f3c0 0x7f343806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.592+0000 7f343e307640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f34380cfa50 0x7f3438162fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34524/0 (socket says 172.21.15.37:34524) 2024-04-26T20:22:13.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.592+0000 7f343e307640 1 -- 172.21.15.37:0/3220681715 learned_addr learned my addr 172.21.15.37:0/3220681715 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:13.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.592+0000 7f343d305640 1 --2- 172.21.15.37:0/3220681715 >> v2:172.21.15.134:3300/0 conn(0x7f34380d1990 0x7f343806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.592+0000 7f343db06640 1 -- 172.21.15.37:0/3220681715 >> v2:172.21.15.134:3300/0 conn(0x7f34380d1990 msgr2=0x7f343806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:13.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.592+0000 7f343db06640 1 --2- 172.21.15.37:0/3220681715 >> v2:172.21.15.134:3300/0 conn(0x7f34380d1990 0x7f343806c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.592+0000 7f343db06640 1 -- 172.21.15.37:0/3220681715 >> v1:172.21.15.37:6789/0 conn(0x7f34380cfa50 legacy=0x7f3438162fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.592+0000 7f343cb04640 1 -- 172.21.15.37:0/3220681715 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4185292226 0 0) 0x7f3438074050 con 0x7f34380cfa50 2024-04-26T20:22:13.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.592+0000 7f343db06640 1 -- 172.21.15.37:0/3220681715 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34380b1620 con 0x7f343815f3c0 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.593+0000 7f343db06640 1 --2- 172.21.15.37:0/3220681715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f343815f3c0 0x7f343806c240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f343806e460 tx=0x7f343002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5c07cbc09a7258 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.593+0000 7f343cb04640 1 -- 172.21.15.37:0/3220681715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430037070 con 0x7f343815f3c0 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.593+0000 7f343cb04640 1 -- 172.21.15.37:0/3220681715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3430032440 con 0x7f343815f3c0 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.593+0000 7f343cb04640 1 -- 172.21.15.37:0/3220681715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343002ae60 con 0x7f343815f3c0 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.593+0000 7f343fd6a640 1 -- 172.21.15.37:0/3220681715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f343815f3c0 msgr2=0x7f343806c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.593+0000 7f343fd6a640 1 --2- 172.21.15.37:0/3220681715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f343815f3c0 0x7f343806c240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f343806e460 tx=0x7f343002a110 comp rx=0 tx=0).stop 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.593+0000 7f343fd6a640 1 -- 172.21.15.37:0/3220681715 shutdown_connections 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.593+0000 7f343fd6a640 1 --2- 172.21.15.37:0/3220681715 >> v2:172.21.15.134:3300/0 conn(0x7f34380d1990 0x7f343806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.594+0000 7f343fd6a640 1 --2- 172.21.15.37:0/3220681715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f343815f3c0 0x7f343806c240 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.594+0000 7f343fd6a640 1 -- 172.21.15.37:0/3220681715 >> 172.21.15.37:0/3220681715 conn(0x7f34380d1e80 msgr2=0x7f34380d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.594+0000 7f343fd6a640 1 -- 172.21.15.37:0/3220681715 shutdown_connections 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.594+0000 7f343fd6a640 1 -- 172.21.15.37:0/3220681715 wait complete. 2024-04-26T20:22:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.595+0000 7f343fd6a640 1 Processor -- start 2024-04-26T20:22:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.595+0000 7f343fd6a640 1 -- start start 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.595+0000 7f343fd6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34380cfa50 0x7f34381e9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.595+0000 7f343fd6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34380d1990 0x7f34380b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.595+0000 7f343fd6a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f343815f3c0 0x7f34381f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.595+0000 7f343fd6a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34380d1530 con 0x7f34380d1990 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.595+0000 7f343fd6a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34380d13b0 con 0x7f34380cfa50 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.595+0000 7f343fd6a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f343815f7a0 con 0x7f343815f3c0 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343d305640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34380d1990 0x7f34380b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343d305640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34380d1990 0x7f34380b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40198/0 (socket says 172.21.15.37:40198) 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343d305640 1 -- 172.21.15.37:0/1323166359 learned_addr learned my addr 172.21.15.37:0/1323166359 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343db06640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34380cfa50 0x7f34381e9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343e307640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f343815f3c0 0x7f34381f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343d305640 1 -- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f343815f3c0 msgr2=0x7f34381f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343d305640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f343815f3c0 0x7f34381f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343d305640 1 -- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34380cfa50 msgr2=0x7f34381e9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343d305640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34380cfa50 0x7f34381e9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343d305640 1 -- 172.21.15.37:0/1323166359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34381eb790 con 0x7f34380d1990 2024-04-26T20:22:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343d305640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34380d1990 0x7f34380b4d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f342c00e870 tx=0x7f342c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f34367fc640 1 -- 172.21.15.37:0/1323166359 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c017070 con 0x7f34380d1990 2024-04-26T20:22:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f34367fc640 1 -- 172.21.15.37:0/1323166359 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f342c00c940 con 0x7f34380d1990 2024-04-26T20:22:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343fd6a640 1 -- 172.21.15.37:0/1323166359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34381ea6f0 con 0x7f34380d1990 2024-04-26T20:22:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f34367fc640 1 -- 172.21.15.37:0/1323166359 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c00cd30 con 0x7f34380d1990 2024-04-26T20:22:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.596+0000 7f343fd6a640 1 -- 172.21.15.37:0/1323166359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34381eaf90 con 0x7f34380d1990 2024-04-26T20:22:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.597+0000 7f343fd6a640 1 -- 172.21.15.37:0/1323166359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343806a360 con 0x7f34380d1990 2024-04-26T20:22:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.598+0000 7f34367fc640 1 -- 172.21.15.37:0/1323166359 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f342c01a070 con 0x7f34380d1990 2024-04-26T20:22:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.598+0000 7f34367fc640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f342406c8f0 0x7f342406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.598+0000 7f34367fc640 1 -- 172.21.15.37:0/1323166359 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f342c08b990 con 0x7f34380d1990 2024-04-26T20:22:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.598+0000 7f343db06640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f342406c8f0 0x7f342406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.599+0000 7f343db06640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f342406c8f0 0x7f342406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3430034000 tx=0x7f343002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:13.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.603+0000 7f34367fc640 1 -- 172.21.15.37:0/1323166359 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34381eb790 con 0x7f34380d1990 2024-04-26T20:22:13.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.786+0000 7f343fd6a640 1 -- 172.21.15.37:0/1323166359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f343806c2b0 con 0x7f34380d1990 2024-04-26T20:22:13.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.787+0000 7f34367fc640 1 -- 172.21.15.37:0/1323166359 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f342c059e50 con 0x7f34380d1990 2024-04-26T20:22:13.788 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:13.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 -- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f342406c8f0 msgr2=0x7f342406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f342406c8f0 0x7f342406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3430034000 tx=0x7f343002e000 comp rx=0 tx=0).stop 2024-04-26T20:22:13.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 -- 172.21.15.37:0/1323166359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34380d1990 msgr2=0x7f34380b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34380d1990 0x7f34380b4d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f342c00e870 tx=0x7f342c00ed30 comp rx=0 tx=0).stop 2024-04-26T20:22:13.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 -- 172.21.15.37:0/1323166359 shutdown_connections 2024-04-26T20:22:13.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f342406c8f0 0x7f342406edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f343815f3c0 0x7f34381f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34380d1990 0x7f34380b4d30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 --2- 172.21.15.37:0/1323166359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34380cfa50 0x7f34381e9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 -- 172.21.15.37:0/1323166359 >> 172.21.15.37:0/1323166359 conn(0x7f34380d1e80 msgr2=0x7f34380b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:13.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 -- 172.21.15.37:0/1323166359 shutdown_connections 2024-04-26T20:22:13.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.788+0000 7f343fd6a640 1 -- 172.21.15.37:0/1323166359 wait complete. 2024-04-26T20:22:13.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.902+0000 7ff37732b640 1 Processor -- start 2024-04-26T20:22:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.902+0000 7ff37732b640 1 -- start start 2024-04-26T20:22:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff37732b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff3700d0540 0x7ff370072a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff37732b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff37006f490 0x7ff3700cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff37732b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff37015be00 con 0x7ff370072f50 2024-04-26T20:22:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff37732b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff37015c0a0 con 0x7ff3700d0540 2024-04-26T20:22:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff37732b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff37015c380 con 0x7ff37006f490 2024-04-26T20:22:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff3750c7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff3700d0540 0x7ff370072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff3750c7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff3700d0540 0x7ff370072a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40214/0 (socket says 172.21.15.37:40214) 2024-04-26T20:22:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff3750c7640 1 -- 172.21.15.37:0/1935939653 learned_addr learned my addr 172.21.15.37:0/1935939653 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff3758c8640 1 --2- 172.21.15.37:0/1935939653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff37006f490 0x7ff3700cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff3750c7640 1 -- 172.21.15.37:0/1935939653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff37006f490 msgr2=0x7ff3700cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff3750c7640 1 --2- 172.21.15.37:0/1935939653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff37006f490 0x7ff3700cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff3750c7640 1 -- 172.21.15.37:0/1935939653 >> v1:172.21.15.134:6789/0 conn(0x7ff370072f50 legacy=0x7ff37006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.903+0000 7ff3750c7640 1 -- 172.21.15.37:0/1935939653 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37015c660 con 0x7ff3700d0540 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.904+0000 7ff3750c7640 1 --2- 172.21.15.37:0/1935939653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3700d0540 0x7ff370072a10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00bb10 tx=0x7ff36c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1cf4019d3c4d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.904+0000 7ff367fff640 1 -- 172.21.15.37:0/1935939653 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c036020 con 0x7ff3700d0540 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.904+0000 7ff367fff640 1 -- 172.21.15.37:0/1935939653 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36c032820 con 0x7ff3700d0540 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.904+0000 7ff367fff640 1 -- 172.21.15.37:0/1935939653 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c032bf0 con 0x7ff3700d0540 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.904+0000 7ff37732b640 1 -- 172.21.15.37:0/1935939653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3700d0540 msgr2=0x7ff370072a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.904+0000 7ff37732b640 1 --2- 172.21.15.37:0/1935939653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3700d0540 0x7ff370072a10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00bb10 tx=0x7ff36c02c920 comp rx=0 tx=0).stop 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.905+0000 7ff37732b640 1 -- 172.21.15.37:0/1935939653 shutdown_connections 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.905+0000 7ff37732b640 1 --2- 172.21.15.37:0/1935939653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff37006f490 0x7ff3700cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.905+0000 7ff37732b640 1 --2- 172.21.15.37:0/1935939653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3700d0540 0x7ff370072a10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.905+0000 7ff37732b640 1 -- 172.21.15.37:0/1935939653 >> 172.21.15.37:0/1935939653 conn(0x7ff3700d0d50 msgr2=0x7ff3700d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.905+0000 7ff37732b640 1 -- 172.21.15.37:0/1935939653 shutdown_connections 2024-04-26T20:22:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.905+0000 7ff37732b640 1 -- 172.21.15.37:0/1935939653 wait complete. 2024-04-26T20:22:13.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.906+0000 7ff37732b640 1 Processor -- start 2024-04-26T20:22:13.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.906+0000 7ff37732b640 1 -- start start 2024-04-26T20:22:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff37732b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff37006f490 0x7ff370151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff37732b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff370072f50 0x7ff3701f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff37732b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3700d0540 0x7ff3701f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff37732b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3700d0aa0 con 0x7ff370072f50 2024-04-26T20:22:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff37732b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3700d0920 con 0x7ff37006f490 2024-04-26T20:22:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff37732b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff37006e250 con 0x7ff3700d0540 2024-04-26T20:22:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3748c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff370072f50 0x7ff3701f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3748c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff370072f50 0x7ff3701f53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40226/0 (socket says 172.21.15.37:40226) 2024-04-26T20:22:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3748c6640 1 -- 172.21.15.37:0/1597717493 learned_addr learned my addr 172.21.15.37:0/1597717493 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3758c8640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3700d0540 0x7ff3701f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3750c7640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff37006f490 0x7ff370151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3748c6640 1 -- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3700d0540 msgr2=0x7ff3701f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3748c6640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3700d0540 0x7ff3701f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3748c6640 1 -- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff37006f490 msgr2=0x7ff370151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3748c6640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff37006f490 0x7ff370151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3748c6640 1 -- 172.21.15.37:0/1597717493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3701f5a30 con 0x7ff370072f50 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3758c8640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3700d0540 0x7ff3701f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3750c7640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff37006f490 0x7ff370151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff3748c6640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff370072f50 0x7ff3701f53b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c430 tx=0x7ff36000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff365ffb640 1 -- 172.21.15.37:0/1597717493 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360017070 con 0x7ff370072f50 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff365ffb640 1 -- 172.21.15.37:0/1597717493 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3600052d0 con 0x7ff370072f50 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff365ffb640 1 -- 172.21.15.37:0/1597717493 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360003850 con 0x7ff370072f50 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.907+0000 7ff37732b640 1 -- 172.21.15.37:0/1597717493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3701f5d10 con 0x7ff370072f50 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.908+0000 7ff37732b640 1 -- 172.21.15.37:0/1597717493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3701f62d0 con 0x7ff370072f50 2024-04-26T20:22:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.909+0000 7ff37732b640 1 -- 172.21.15.37:0/1597717493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37006a360 con 0x7ff370072f50 2024-04-26T20:22:13.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.909+0000 7ff365ffb640 1 -- 172.21.15.37:0/1597717493 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff36001a070 con 0x7ff370072f50 2024-04-26T20:22:13.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.909+0000 7ff365ffb640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff35406c8f0 0x7ff35406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.909+0000 7ff365ffb640 1 -- 172.21.15.37:0/1597717493 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff36008af00 con 0x7ff370072f50 2024-04-26T20:22:13.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.910+0000 7ff3750c7640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff35406c8f0 0x7ff35406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:13.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.910+0000 7ff3750c7640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff35406c8f0 0x7ff35406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff36c037000 tx=0x7ff36c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:13.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:13.914+0000 7ff365ffb640 1 -- 172.21.15.37:0/1597717493 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff360059320 con 0x7ff370072f50 2024-04-26T20:22:14.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.101+0000 7ff37732b640 1 -- 172.21.15.37:0/1597717493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff37006f8e0 con 0x7ff370072f50 2024-04-26T20:22:14.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.102+0000 7ff365ffb640 1 -- 172.21.15.37:0/1597717493 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7ff360058c20 con 0x7ff370072f50 2024-04-26T20:22:14.103 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:14.103 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:14.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 -- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff35406c8f0 msgr2=0x7ff35406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff35406c8f0 0x7ff35406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff36c037000 tx=0x7ff36c008040 comp rx=0 tx=0).stop 2024-04-26T20:22:14.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 -- 172.21.15.37:0/1597717493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff370072f50 msgr2=0x7ff3701f53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff370072f50 0x7ff3701f53b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c430 tx=0x7ff36000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 -- 172.21.15.37:0/1597717493 shutdown_connections 2024-04-26T20:22:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff35406c8f0 0x7ff35406edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3700d0540 0x7ff3701f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff370072f50 0x7ff3701f53b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 --2- 172.21.15.37:0/1597717493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff37006f490 0x7ff370151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 -- 172.21.15.37:0/1597717493 >> 172.21.15.37:0/1597717493 conn(0x7ff3700d0d50 msgr2=0x7ff37015dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 -- 172.21.15.37:0/1597717493 shutdown_connections 2024-04-26T20:22:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.104+0000 7ff37732b640 1 -- 172.21.15.37:0/1597717493 wait complete. 2024-04-26T20:22:14.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:14.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:14.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.220+0000 7fcb63aea640 1 Processor -- start 2024-04-26T20:22:14.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.220+0000 7fcb63aea640 1 -- start start 2024-04-26T20:22:14.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb63aea640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcb5c15f400 0x7fcb5c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb63aea640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcb5c0d5a80 0x7fcb5c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb63aea640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb5c0b2760 con 0x7fcb5c0b4b20 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb63aea640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fcb5c0b43e0 con 0x7fcb5c0d5a80 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb63aea640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcb5c0b2a00 con 0x7fcb5c15f400 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb61085640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcb5c0d5a80 0x7fcb5c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb62087640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fcb5c0b4b20 0x7fcb5c0d3420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34526/0 (socket says 172.21.15.37:34526) 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb61085640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcb5c0d5a80 0x7fcb5c0b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40232/0 (socket says 172.21.15.37:40232) 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb62087640 1 -- 172.21.15.37:0/795876112 learned_addr learned my addr 172.21.15.37:0/795876112 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb61886640 1 --2- 172.21.15.37:0/795876112 >> v2:172.21.15.134:3300/0 conn(0x7fcb5c15f400 0x7fcb5c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb61085640 1 -- 172.21.15.37:0/795876112 >> v2:172.21.15.134:3300/0 conn(0x7fcb5c15f400 msgr2=0x7fcb5c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb61085640 1 --2- 172.21.15.37:0/795876112 >> v2:172.21.15.134:3300/0 conn(0x7fcb5c15f400 0x7fcb5c0b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb61085640 1 -- 172.21.15.37:0/795876112 >> v1:172.21.15.37:6789/0 conn(0x7fcb5c0b4b20 legacy=0x7fcb5c0d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb61085640 1 -- 172.21.15.37:0/795876112 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c0b2ce0 con 0x7fcb5c0d5a80 2024-04-26T20:22:14.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.221+0000 7fcb60884640 1 -- 172.21.15.37:0/795876112 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3734401029 0 0) 0x7fcb5c0b2760 con 0x7fcb5c0b4b20 2024-04-26T20:22:14.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.222+0000 7fcb61085640 1 --2- 172.21.15.37:0/795876112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb5c0d5a80 0x7fcb5c0b45e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c06cc90 tx=0x7fcb4c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d20fe534dbc773 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:14.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.222+0000 7fcb60884640 1 -- 172.21.15.37:0/795876112 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c037070 con 0x7fcb5c0d5a80 2024-04-26T20:22:14.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.222+0000 7fcb60884640 1 -- 172.21.15.37:0/795876112 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb4c032440 con 0x7fcb5c0d5a80 2024-04-26T20:22:14.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.222+0000 7fcb60884640 1 -- 172.21.15.37:0/795876112 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c02ae60 con 0x7fcb5c0d5a80 2024-04-26T20:22:14.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.223+0000 7fcb63aea640 1 -- 172.21.15.37:0/795876112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb5c0d5a80 msgr2=0x7fcb5c0b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.223+0000 7fcb63aea640 1 --2- 172.21.15.37:0/795876112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb5c0d5a80 0x7fcb5c0b45e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c06cc90 tx=0x7fcb4c02a110 comp rx=0 tx=0).stop 2024-04-26T20:22:14.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.223+0000 7fcb63aea640 1 -- 172.21.15.37:0/795876112 shutdown_connections 2024-04-26T20:22:14.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.223+0000 7fcb63aea640 1 --2- 172.21.15.37:0/795876112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb5c0d5a80 0x7fcb5c0b45e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.223+0000 7fcb63aea640 1 --2- 172.21.15.37:0/795876112 >> v2:172.21.15.134:3300/0 conn(0x7fcb5c15f400 0x7fcb5c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.223+0000 7fcb63aea640 1 -- 172.21.15.37:0/795876112 >> 172.21.15.37:0/795876112 conn(0x7fcb5c09e4f0 msgr2=0x7fcb5c0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:14.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.223+0000 7fcb63aea640 1 -- 172.21.15.37:0/795876112 shutdown_connections 2024-04-26T20:22:14.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.223+0000 7fcb63aea640 1 -- 172.21.15.37:0/795876112 wait complete. 2024-04-26T20:22:14.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.224+0000 7fcb63aea640 1 Processor -- start 2024-04-26T20:22:14.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.224+0000 7fcb63aea640 1 -- start start 2024-04-26T20:22:14.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb63aea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb5c0b4b20 0x7fcb5c0cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb63aea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c0d5a80 0x7fcb5c14fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb63aea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb5c15f400 0x7fcb5c1f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb63aea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c0b4f30 con 0x7fcb5c15f400 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb63aea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c0b32a0 con 0x7fcb5c0b4b20 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb63aea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5c0b50b0 con 0x7fcb5c0d5a80 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb62087640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb5c15f400 0x7fcb5c1f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb62087640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb5c15f400 0x7fcb5c1f5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40240/0 (socket says 172.21.15.37:40240) 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb62087640 1 -- 172.21.15.37:0/2603225136 learned_addr learned my addr 172.21.15.37:0/2603225136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb61886640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb5c0b4b20 0x7fcb5c0cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb61085640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c0d5a80 0x7fcb5c14fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb62087640 1 -- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c0d5a80 msgr2=0x7fcb5c14fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb62087640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c0d5a80 0x7fcb5c14fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb62087640 1 -- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb5c0b4b20 msgr2=0x7fcb5c0cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb62087640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb5c0b4b20 0x7fcb5c0cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb62087640 1 -- 172.21.15.37:0/2603225136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c150c90 con 0x7fcb5c15f400 2024-04-26T20:22:14.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb61886640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb5c0b4b20 0x7fcb5c0cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb61085640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c0d5a80 0x7fcb5c14fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb62087640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb5c15f400 0x7fcb5c1f5d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400cb20 tx=0x7fcb540021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb527fc640 1 -- 172.21.15.37:0/2603225136 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400ac00 con 0x7fcb5c15f400 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb527fc640 1 -- 172.21.15.37:0/2603225136 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb5400adc0 con 0x7fcb5c15f400 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb527fc640 1 -- 172.21.15.37:0/2603225136 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54012640 con 0x7fcb5c15f400 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb63aea640 1 -- 172.21.15.37:0/2603225136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5c1506d0 con 0x7fcb5c15f400 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.225+0000 7fcb63aea640 1 -- 172.21.15.37:0/2603225136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5c1f5500 con 0x7fcb5c15f400 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.226+0000 7fcb63aea640 1 -- 172.21.15.37:0/2603225136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5c1f5830 con 0x7fcb5c15f400 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.227+0000 7fcb527fc640 1 -- 172.21.15.37:0/2603225136 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb540127e0 con 0x7fcb5c15f400 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.227+0000 7fcb527fc640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb4006c8f0 0x7fcb4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.227+0000 7fcb527fc640 1 -- 172.21.15.37:0/2603225136 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcb5408c1f0 con 0x7fcb5c15f400 2024-04-26T20:22:14.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.228+0000 7fcb61886640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb4006c8f0 0x7fcb4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.228+0000 7fcb61886640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb4006c8f0 0x7fcb4006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb58006fd0 tx=0x7fcb58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:14.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.232+0000 7fcb527fc640 1 -- 172.21.15.37:0/2603225136 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb5405a5a0 con 0x7fcb5c15f400 2024-04-26T20:22:14.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.414+0000 7fcb63aea640 1 -- 172.21.15.37:0/2603225136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcb5c06a360 con 0x7fcb5c15f400 2024-04-26T20:22:14.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.415+0000 7fcb527fc640 1 -- 172.21.15.37:0/2603225136 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fcb54059ea0 con 0x7fcb5c15f400 2024-04-26T20:22:14.416 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:14.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.416+0000 7fcb63aea640 1 -- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb4006c8f0 msgr2=0x7fcb4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.416+0000 7fcb63aea640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb4006c8f0 0x7fcb4006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb58006fd0 tx=0x7fcb58008040 comp rx=0 tx=0).stop 2024-04-26T20:22:14.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.416+0000 7fcb63aea640 1 -- 172.21.15.37:0/2603225136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb5c15f400 msgr2=0x7fcb5c1f5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.416+0000 7fcb63aea640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb5c15f400 0x7fcb5c1f5d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400cb20 tx=0x7fcb540021c0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.417+0000 7fcb63aea640 1 -- 172.21.15.37:0/2603225136 shutdown_connections 2024-04-26T20:22:14.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.417+0000 7fcb63aea640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb5c15f400 0x7fcb5c1f5d00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.417+0000 7fcb63aea640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb4006c8f0 0x7fcb4006edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.417+0000 7fcb63aea640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c0d5a80 0x7fcb5c14fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.417+0000 7fcb63aea640 1 --2- 172.21.15.37:0/2603225136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb5c0b4b20 0x7fcb5c0cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.417+0000 7fcb63aea640 1 -- 172.21.15.37:0/2603225136 >> 172.21.15.37:0/2603225136 conn(0x7fcb5c09e4f0 msgr2=0x7fcb5c06e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:14.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.417+0000 7fcb63aea640 1 -- 172.21.15.37:0/2603225136 shutdown_connections 2024-04-26T20:22:14.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.417+0000 7fcb63aea640 1 -- 172.21.15.37:0/2603225136 wait complete. 2024-04-26T20:22:14.428 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.531+0000 7f489f146640 1 Processor -- start 2024-04-26T20:22:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.531+0000 7f489f146640 1 -- start start 2024-04-26T20:22:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.532+0000 7f489f146640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4898160800 0x7f48981610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.532+0000 7f489f146640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f48980d5b60 0x7f48980d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.532+0000 7f489f146640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48980d0930 con 0x7f48980d2150 2024-04-26T20:22:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.532+0000 7f489f146640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f48980d25c0 con 0x7f48980d5b60 2024-04-26T20:22:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.532+0000 7f489f146640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f48980d0c00 con 0x7f4898160800 2024-04-26T20:22:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.532+0000 7f4897fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f48980d5b60 0x7f48980d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.532+0000 7f4897fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f48980d5b60 0x7f48980d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40252/0 (socket says 172.21.15.37:40252) 2024-04-26T20:22:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.532+0000 7f4897fff640 1 -- 172.21.15.37:0/763744506 learned_addr learned my addr 172.21.15.37:0/763744506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.532+0000 7f489cee2640 1 --2- 172.21.15.37:0/763744506 >> v2:172.21.15.134:3300/0 conn(0x7f4898160800 0x7f48981610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.533+0000 7f4897fff640 1 -- 172.21.15.37:0/763744506 >> v2:172.21.15.134:3300/0 conn(0x7f4898160800 msgr2=0x7f48981610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.533+0000 7f4897fff640 1 --2- 172.21.15.37:0/763744506 >> v2:172.21.15.134:3300/0 conn(0x7f4898160800 0x7f48981610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.533+0000 7f4897fff640 1 -- 172.21.15.37:0/763744506 >> v1:172.21.15.134:6789/0 conn(0x7f48980d2150 legacy=0x7f48980d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.533+0000 7f4897fff640 1 -- 172.21.15.37:0/763744506 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48980d0ee0 con 0x7f48980d5b60 2024-04-26T20:22:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.533+0000 7f4897fff640 1 --2- 172.21.15.37:0/763744506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48980d5b60 0x7f48980d1c10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f489806c5f0 tx=0x7f488002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272a5f0480df7046 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.534+0000 7f48977fe640 1 -- 172.21.15.37:0/763744506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880037070 con 0x7f48980d5b60 2024-04-26T20:22:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.534+0000 7f48977fe640 1 -- 172.21.15.37:0/763744506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4880032440 con 0x7f48980d5b60 2024-04-26T20:22:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.534+0000 7f48977fe640 1 -- 172.21.15.37:0/763744506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880032810 con 0x7f48980d5b60 2024-04-26T20:22:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.534+0000 7f489f146640 1 -- 172.21.15.37:0/763744506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48980d5b60 msgr2=0x7f48980d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.534+0000 7f489f146640 1 --2- 172.21.15.37:0/763744506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48980d5b60 0x7f48980d1c10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f489806c5f0 tx=0x7f488002a270 comp rx=0 tx=0).stop 2024-04-26T20:22:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.534+0000 7f489f146640 1 -- 172.21.15.37:0/763744506 shutdown_connections 2024-04-26T20:22:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.534+0000 7f489f146640 1 --2- 172.21.15.37:0/763744506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48980d5b60 0x7f48980d1c10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.534+0000 7f489f146640 1 --2- 172.21.15.37:0/763744506 >> v2:172.21.15.134:3300/0 conn(0x7f4898160800 0x7f48981610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.534+0000 7f489f146640 1 -- 172.21.15.37:0/763744506 >> 172.21.15.37:0/763744506 conn(0x7f489809e550 msgr2=0x7f48980d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.534+0000 7f489f146640 1 -- 172.21.15.37:0/763744506 shutdown_connections 2024-04-26T20:22:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.534+0000 7f489f146640 1 -- 172.21.15.37:0/763744506 wait complete. 2024-04-26T20:22:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.535+0000 7f489f146640 1 Processor -- start 2024-04-26T20:22:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.535+0000 7f489f146640 1 -- start start 2024-04-26T20:22:14.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.536+0000 7f489f146640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48980d2150 0x7f4898154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.536+0000 7f489f146640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48980d5b60 0x7f48980b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.536+0000 7f489f146640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4898160800 0x7f48980b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.536+0000 7f489f146640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f48980d30f0 con 0x7f4898160800 2024-04-26T20:22:14.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.536+0000 7f489f146640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48980d2e10 con 0x7f48980d2150 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.536+0000 7f489f146640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f48980b6300 con 0x7f48980d5b60 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.536+0000 7f489d6e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4898160800 0x7f48980b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.536+0000 7f489d6e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4898160800 0x7f48980b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40258/0 (socket says 172.21.15.37:40258) 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.536+0000 7f489d6e3640 1 -- 172.21.15.37:0/4288921179 learned_addr learned my addr 172.21.15.37:0/4288921179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.536+0000 7f4897fff640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48980d5b60 0x7f48980b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.536+0000 7f489cee2640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48980d2150 0x7f4898154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.537+0000 7f489d6e3640 1 -- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48980d5b60 msgr2=0x7f48980b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.537+0000 7f489d6e3640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48980d5b60 0x7f48980b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.537+0000 7f489d6e3640 1 -- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48980d2150 msgr2=0x7f4898154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.537+0000 7f489d6e3640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48980d2150 0x7f4898154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.537+0000 7f489d6e3640 1 -- 172.21.15.37:0/4288921179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4898154b80 con 0x7f4898160800 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.537+0000 7f489d6e3640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4898160800 0x7f48980b45e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f488800c3c0 tx=0x7f488800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.537+0000 7f48957fa640 1 -- 172.21.15.37:0/4288921179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888017070 con 0x7f4898160800 2024-04-26T20:22:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.537+0000 7f48957fa640 1 -- 172.21.15.37:0/4288921179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48880052d0 con 0x7f4898160800 2024-04-26T20:22:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.537+0000 7f489f146640 1 -- 172.21.15.37:0/4288921179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48980cdac0 con 0x7f4898160800 2024-04-26T20:22:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.537+0000 7f48957fa640 1 -- 172.21.15.37:0/4288921179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888003830 con 0x7f4898160800 2024-04-26T20:22:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.537+0000 7f489f146640 1 -- 172.21.15.37:0/4288921179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48980b21c0 con 0x7f4898160800 2024-04-26T20:22:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.538+0000 7f489f146640 1 -- 172.21.15.37:0/4288921179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489806a360 con 0x7f4898160800 2024-04-26T20:22:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.538+0000 7f48957fa640 1 -- 172.21.15.37:0/4288921179 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f488801a070 con 0x7f4898160800 2024-04-26T20:22:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.539+0000 7f48957fa640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f487806c8f0 0x7f487806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.539+0000 7f48957fa640 1 -- 172.21.15.37:0/4288921179 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f488808ac00 con 0x7f4898160800 2024-04-26T20:22:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.539+0000 7f489cee2640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f487806c8f0 0x7f487806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.540+0000 7f489cee2640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f487806c8f0 0x7f487806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48980cedb0 tx=0x7f489000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.543+0000 7f48957fa640 1 -- 172.21.15.37:0/4288921179 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4888091080 con 0x7f4898160800 2024-04-26T20:22:14.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.727+0000 7f489f146640 1 -- 172.21.15.37:0/4288921179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48980d4530 con 0x7f4898160800 2024-04-26T20:22:14.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.727+0000 7f48957fa640 1 -- 172.21.15.37:0/4288921179 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f4888058a70 con 0x7f4898160800 2024-04-26T20:22:14.729 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 -- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f487806c8f0 msgr2=0x7f487806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f487806c8f0 0x7f487806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48980cedb0 tx=0x7f489000b040 comp rx=0 tx=0).stop 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 -- 172.21.15.37:0/4288921179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4898160800 msgr2=0x7f48980b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4898160800 0x7f48980b45e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f488800c3c0 tx=0x7f488800c880 comp rx=0 tx=0).stop 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 -- 172.21.15.37:0/4288921179 shutdown_connections 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4898160800 0x7f48980b45e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f487806c8f0 0x7f487806edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48980d5b60 0x7f48980b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 --2- 172.21.15.37:0/4288921179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48980d2150 0x7f4898154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 -- 172.21.15.37:0/4288921179 >> 172.21.15.37:0/4288921179 conn(0x7f489809e550 msgr2=0x7f489815f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 -- 172.21.15.37:0/4288921179 shutdown_connections 2024-04-26T20:22:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.729+0000 7f489f146640 1 -- 172.21.15.37:0/4288921179 wait complete. 2024-04-26T20:22:14.741 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.844+0000 7f79e7ada640 1 Processor -- start 2024-04-26T20:22:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.844+0000 7f79e7ada640 1 -- start start 2024-04-26T20:22:14.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e7ada640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00d5a80 0x7f79e00b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e7ada640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e00b2760 con 0x7f79e015f400 2024-04-26T20:22:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e7ada640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e00b2a00 con 0x7f79e00b4b20 2024-04-26T20:22:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e7ada640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f79e00b43e0 con 0x7f79e00d5a80 2024-04-26T20:22:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e5876640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f79e015f400 0x7f79e00b37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34528/0 (socket says 172.21.15.37:34528) 2024-04-26T20:22:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e5876640 1 -- 172.21.15.37:0/2982471058 learned_addr learned my addr 172.21.15.37:0/2982471058 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e5075640 1 --2- 172.21.15.37:0/2982471058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00d5a80 0x7f79e00b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e4874640 1 -- 172.21.15.37:0/2982471058 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3922763989 0 0) 0x7f79e00b2760 con 0x7f79e015f400 2024-04-26T20:22:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e4874640 1 -- 172.21.15.37:0/2982471058 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79cc003520 con 0x7f79e015f400 2024-04-26T20:22:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e5075640 1 -- 172.21.15.37:0/2982471058 >> v1:172.21.15.134:6789/0 conn(0x7f79e00b4b20 legacy=0x7f79e01630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e4874640 1 -- 172.21.15.37:0/2982471058 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1378396303 0 0) 0x7f79cc003520 con 0x7f79e015f400 2024-04-26T20:22:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e5075640 1 -- 172.21.15.37:0/2982471058 >> v1:172.21.15.37:6789/0 conn(0x7f79e015f400 legacy=0x7f79e00b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.845+0000 7f79e5075640 1 -- 172.21.15.37:0/2982471058 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e00b2ce0 con 0x7f79e00d5a80 2024-04-26T20:22:14.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.846+0000 7f79e5075640 1 --2- 172.21.15.37:0/2982471058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00d5a80 0x7f79e00b45e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79d00097d0 tx=0x7f79d002a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de34dfe3c9ddd853 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:14.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.846+0000 7f79e4874640 1 -- 172.21.15.37:0/2982471058 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0037070 con 0x7f79e00d5a80 2024-04-26T20:22:14.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.846+0000 7f79e4874640 1 -- 172.21.15.37:0/2982471058 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d002ab40 con 0x7f79e00d5a80 2024-04-26T20:22:14.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.846+0000 7f79e4874640 1 -- 172.21.15.37:0/2982471058 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc001670 con 0x7f79e00d5a80 2024-04-26T20:22:14.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.847+0000 7f79e7ada640 1 -- 172.21.15.37:0/2982471058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00d5a80 msgr2=0x7f79e00b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.847+0000 7f79e7ada640 1 --2- 172.21.15.37:0/2982471058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00d5a80 0x7f79e00b45e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79d00097d0 tx=0x7f79d002a0b0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.847+0000 7f79e7ada640 1 -- 172.21.15.37:0/2982471058 shutdown_connections 2024-04-26T20:22:14.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.847+0000 7f79e7ada640 1 --2- 172.21.15.37:0/2982471058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00d5a80 0x7f79e00b45e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.847+0000 7f79e7ada640 1 -- 172.21.15.37:0/2982471058 >> 172.21.15.37:0/2982471058 conn(0x7f79e009e4f0 msgr2=0x7f79e00d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:14.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.847+0000 7f79e7ada640 1 -- 172.21.15.37:0/2982471058 shutdown_connections 2024-04-26T20:22:14.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.847+0000 7f79e7ada640 1 -- 172.21.15.37:0/2982471058 wait complete. 2024-04-26T20:22:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.848+0000 7f79e7ada640 1 Processor -- start 2024-04-26T20:22:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.848+0000 7f79e7ada640 1 -- start start 2024-04-26T20:22:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e7ada640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e00b4b20 0x7f79e01559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e7ada640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00d5a80 0x7f79e00cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e7ada640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79e015f400 0x7f79e009f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e7ada640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79e00b32a0 con 0x7f79e00b4b20 2024-04-26T20:22:14.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e7ada640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79e00b3ed0 con 0x7f79e015f400 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e7ada640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f79e00b4f70 con 0x7f79e00d5a80 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e5876640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e00b4b20 0x7f79e01559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e5876640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e00b4b20 0x7f79e01559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40266/0 (socket says 172.21.15.37:40266) 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e5876640 1 -- 172.21.15.37:0/3115216372 learned_addr learned my addr 172.21.15.37:0/3115216372 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e5075640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00d5a80 0x7f79e00cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e5876640 1 -- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00d5a80 msgr2=0x7f79e00cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e5876640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00d5a80 0x7f79e00cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e5876640 1 -- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79e015f400 msgr2=0x7f79e009f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e5876640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79e015f400 0x7f79e009f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e5876640 1 -- 172.21.15.37:0/3115216372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e00cd9f0 con 0x7f79e00b4b20 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.849+0000 7f79e5876640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e00b4b20 0x7f79e01559b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00c520 tx=0x7f79dc00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.850+0000 7f79d67fc640 1 -- 172.21.15.37:0/3115216372 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc018070 con 0x7f79e00b4b20 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.850+0000 7f79d67fc640 1 -- 172.21.15.37:0/3115216372 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79dc0074e0 con 0x7f79e00b4b20 2024-04-26T20:22:14.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.850+0000 7f79e7ada640 1 -- 172.21.15.37:0/3115216372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e00cdcd0 con 0x7f79e00b4b20 2024-04-26T20:22:14.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.850+0000 7f79d67fc640 1 -- 172.21.15.37:0/3115216372 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc004a20 con 0x7f79e00b4b20 2024-04-26T20:22:14.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.850+0000 7f79e7ada640 1 -- 172.21.15.37:0/3115216372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e00ce290 con 0x7f79e00b4b20 2024-04-26T20:22:14.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.851+0000 7f79e7ada640 1 -- 172.21.15.37:0/3115216372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e006a360 con 0x7f79e00b4b20 2024-04-26T20:22:14.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.851+0000 7f79d67fc640 1 -- 172.21.15.37:0/3115216372 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79dc01b070 con 0x7f79e00b4b20 2024-04-26T20:22:14.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.852+0000 7f79d67fc640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f79c406c8f0 0x7f79c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:14.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.852+0000 7f79d67fc640 1 -- 172.21.15.37:0/3115216372 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f79dc08af40 con 0x7f79e00b4b20 2024-04-26T20:22:14.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.852+0000 7f79e5075640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f79c406c8f0 0x7f79c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.853+0000 7f79e5075640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f79c406c8f0 0x7f79c406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f79d0034000 tx=0x7f79d002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:14.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:14.856+0000 7f79d67fc640 1 -- 172.21.15.37:0/3115216372 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f79dc059370 con 0x7f79e00b4b20 2024-04-26T20:22:15.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.040+0000 7f79e7ada640 1 -- 172.21.15.37:0/3115216372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f79e00bad40 con 0x7f79e00b4b20 2024-04-26T20:22:15.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.041+0000 7f79d67fc640 1 -- 172.21.15.37:0/3115216372 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f79dc05f020 con 0x7f79e00b4b20 2024-04-26T20:22:15.042 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.042+0000 7f79e7ada640 1 -- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f79c406c8f0 msgr2=0x7f79c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.042+0000 7f79e7ada640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f79c406c8f0 0x7f79c406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f79d0034000 tx=0x7f79d002e000 comp rx=0 tx=0).stop 2024-04-26T20:22:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.042+0000 7f79e7ada640 1 -- 172.21.15.37:0/3115216372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e00b4b20 msgr2=0x7f79e01559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.042+0000 7f79e7ada640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e00b4b20 0x7f79e01559b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00c520 tx=0x7f79dc00c9e0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.043+0000 7f79e7ada640 1 -- 172.21.15.37:0/3115216372 shutdown_connections 2024-04-26T20:22:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.043+0000 7f79e7ada640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79e015f400 0x7f79e009f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.043+0000 7f79e7ada640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f79c406c8f0 0x7f79c406edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.043+0000 7f79e7ada640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00d5a80 0x7f79e00cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.043+0000 7f79e7ada640 1 --2- 172.21.15.37:0/3115216372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e00b4b20 0x7f79e01559b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.043+0000 7f79e7ada640 1 -- 172.21.15.37:0/3115216372 >> 172.21.15.37:0/3115216372 conn(0x7f79e009e4f0 msgr2=0x7f79e00b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.043+0000 7f79e7ada640 1 -- 172.21.15.37:0/3115216372 shutdown_connections 2024-04-26T20:22:15.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.043+0000 7f79e7ada640 1 -- 172.21.15.37:0/3115216372 wait complete. 2024-04-26T20:22:15.055 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:15.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.153+0000 7f9334f39640 1 Processor -- start 2024-04-26T20:22:15.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.153+0000 7f9334f39640 1 -- start start 2024-04-26T20:22:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.154+0000 7f9334f39640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93300d3820 0x7f93300d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.154+0000 7f9334f39640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93300b4ba0 0x7f93300b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.154+0000 7f9334f39640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93300d0210 con 0x7f9330162b40 2024-04-26T20:22:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.154+0000 7f9334f39640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f93300d41c0 con 0x7f93300d3820 2024-04-26T20:22:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.154+0000 7f9334f39640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93300d04f0 con 0x7f93300b4ba0 2024-04-26T20:22:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.154+0000 7f932e59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9330162b40 0x7f93300d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34532/0 (socket says 172.21.15.37:34532) 2024-04-26T20:22:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.154+0000 7f932e59c640 1 -- 172.21.15.37:0/2311300879 learned_addr learned my addr 172.21.15.37:0/2311300879 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.155+0000 7f932dd9b640 1 --2- 172.21.15.37:0/2311300879 >> v2:172.21.15.134:3300/0 conn(0x7f93300d3820 0x7f93300d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.155+0000 7f932ed9d640 1 --2- 172.21.15.37:0/2311300879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93300b4ba0 0x7f93300b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.155+0000 7f932d59a640 1 -- 172.21.15.37:0/2311300879 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 80224681 0 0) 0x7f93300d0210 con 0x7f9330162b40 2024-04-26T20:22:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.155+0000 7f932d59a640 1 -- 172.21.15.37:0/2311300879 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f931c003520 con 0x7f9330162b40 2024-04-26T20:22:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.155+0000 7f932ed9d640 1 -- 172.21.15.37:0/2311300879 >> v2:172.21.15.134:3300/0 conn(0x7f93300d3820 msgr2=0x7f93300d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.155+0000 7f932ed9d640 1 --2- 172.21.15.37:0/2311300879 >> v2:172.21.15.134:3300/0 conn(0x7f93300d3820 0x7f93300d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.155+0000 7f932ed9d640 1 -- 172.21.15.37:0/2311300879 >> v1:172.21.15.37:6789/0 conn(0x7f9330162b40 legacy=0x7f93300d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.155+0000 7f932ed9d640 1 -- 172.21.15.37:0/2311300879 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93300d07d0 con 0x7f93300b4ba0 2024-04-26T20:22:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.156+0000 7f932ed9d640 1 --2- 172.21.15.37:0/2311300879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93300b4ba0 0x7f93300b5030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9320009d90 tx=0x7f932002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36fa132c5c1a0c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.156+0000 7f932d59a640 1 -- 172.21.15.37:0/2311300879 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320007740 con 0x7f93300b4ba0 2024-04-26T20:22:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.156+0000 7f932d59a640 1 -- 172.21.15.37:0/2311300879 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93200078e0 con 0x7f93300b4ba0 2024-04-26T20:22:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.156+0000 7f932d59a640 1 -- 172.21.15.37:0/2311300879 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320007c90 con 0x7f93300b4ba0 2024-04-26T20:22:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.156+0000 7f9334f39640 1 -- 172.21.15.37:0/2311300879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93300b4ba0 msgr2=0x7f93300b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.156+0000 7f9334f39640 1 --2- 172.21.15.37:0/2311300879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93300b4ba0 0x7f93300b5030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9320009d90 tx=0x7f932002a660 comp rx=0 tx=0).stop 2024-04-26T20:22:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.157+0000 7f9334f39640 1 -- 172.21.15.37:0/2311300879 shutdown_connections 2024-04-26T20:22:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.157+0000 7f9334f39640 1 --2- 172.21.15.37:0/2311300879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93300b4ba0 0x7f93300b5030 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.157+0000 7f9334f39640 1 --2- 172.21.15.37:0/2311300879 >> v2:172.21.15.134:3300/0 conn(0x7f93300d3820 0x7f93300d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.157+0000 7f9334f39640 1 -- 172.21.15.37:0/2311300879 >> 172.21.15.37:0/2311300879 conn(0x7f933009e550 msgr2=0x7f93300d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.157+0000 7f9334f39640 1 -- 172.21.15.37:0/2311300879 shutdown_connections 2024-04-26T20:22:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.157+0000 7f9334f39640 1 -- 172.21.15.37:0/2311300879 wait complete. 2024-04-26T20:22:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.158+0000 7f9334f39640 1 Processor -- start 2024-04-26T20:22:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.158+0000 7f9334f39640 1 -- start start 2024-04-26T20:22:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f9334f39640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93300b4ba0 0x7f93300c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f9334f39640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93300d3820 0x7f93301545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f9334f39640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9330162b40 0x7f93300ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f9334f39640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f93300d12b0 con 0x7f93300d3820 2024-04-26T20:22:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f9334f39640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93300d0d90 con 0x7f93300b4ba0 2024-04-26T20:22:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f9334f39640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93300cfe30 con 0x7f9330162b40 2024-04-26T20:22:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f932dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93300d3820 0x7f93301545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f932dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93300d3820 0x7f93301545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40272/0 (socket says 172.21.15.37:40272) 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f932dd9b640 1 -- 172.21.15.37:0/2313315038 learned_addr learned my addr 172.21.15.37:0/2313315038 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f932e59c640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93300b4ba0 0x7f93300c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f932ed9d640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9330162b40 0x7f93300ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f932dd9b640 1 -- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9330162b40 msgr2=0x7f93300ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f932dd9b640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9330162b40 0x7f93300ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f932dd9b640 1 -- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93300b4ba0 msgr2=0x7f93300c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f932dd9b640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93300b4ba0 0x7f93300c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f932dd9b640 1 -- 172.21.15.37:0/2313315038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9330154ae0 con 0x7f93300d3820 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f932dd9b640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93300d3820 0x7f93301545a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f931800e9b0 tx=0x7f931800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f9326ffd640 1 -- 172.21.15.37:0/2313315038 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318017070 con 0x7f93300d3820 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f9326ffd640 1 -- 172.21.15.37:0/2313315038 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931800c9a0 con 0x7f93300d3820 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.159+0000 7f9326ffd640 1 -- 172.21.15.37:0/2313315038 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931800cd90 con 0x7f93300d3820 2024-04-26T20:22:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.160+0000 7f9334f39640 1 -- 172.21.15.37:0/2313315038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93300c8fb0 con 0x7f93300d3820 2024-04-26T20:22:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.160+0000 7f9334f39640 1 -- 172.21.15.37:0/2313315038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93300c9570 con 0x7f93300d3820 2024-04-26T20:22:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.161+0000 7f9334f39640 1 -- 172.21.15.37:0/2313315038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f933006a360 con 0x7f93300d3820 2024-04-26T20:22:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.161+0000 7f9326ffd640 1 -- 172.21.15.37:0/2313315038 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f931801a070 con 0x7f93300d3820 2024-04-26T20:22:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.162+0000 7f9326ffd640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f931006c8f0 0x7f931006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.162+0000 7f9326ffd640 1 -- 172.21.15.37:0/2313315038 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f931808bb60 con 0x7f93300d3820 2024-04-26T20:22:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.162+0000 7f932e59c640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f931006c8f0 0x7f931006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.162+0000 7f932e59c640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f931006c8f0 0x7f931006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9328005880 tx=0x7f9328009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.166+0000 7f9326ffd640 1 -- 172.21.15.37:0/2313315038 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9318059f90 con 0x7f93300d3820 2024-04-26T20:22:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.352+0000 7f9334f39640 1 -- 172.21.15.37:0/2313315038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93300d2960 con 0x7f93300d3820 2024-04-26T20:22:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.353+0000 7f9326ffd640 1 -- 172.21.15.37:0/2313315038 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f9318059890 con 0x7f93300d3820 2024-04-26T20:22:15.354 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:15.354 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:15.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 -- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f931006c8f0 msgr2=0x7f931006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f931006c8f0 0x7f931006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9328005880 tx=0x7f9328009040 comp rx=0 tx=0).stop 2024-04-26T20:22:15.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 -- 172.21.15.37:0/2313315038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93300d3820 msgr2=0x7f93301545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93300d3820 0x7f93301545a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f931800e9b0 tx=0x7f931800ee70 comp rx=0 tx=0).stop 2024-04-26T20:22:15.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 -- 172.21.15.37:0/2313315038 shutdown_connections 2024-04-26T20:22:15.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f931006c8f0 0x7f931006edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9330162b40 0x7f93300ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93300d3820 0x7f93301545a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 --2- 172.21.15.37:0/2313315038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93300b4ba0 0x7f93300c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 -- 172.21.15.37:0/2313315038 >> 172.21.15.37:0/2313315038 conn(0x7f933009e550 msgr2=0x7f933006d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:15.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 -- 172.21.15.37:0/2313315038 shutdown_connections 2024-04-26T20:22:15.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.355+0000 7f9334f39640 1 -- 172.21.15.37:0/2313315038 wait complete. 2024-04-26T20:22:15.367 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:15.367 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:15.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.470+0000 7f35d9237640 1 Processor -- start 2024-04-26T20:22:15.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.470+0000 7f35d9237640 1 -- start start 2024-04-26T20:22:15.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.470+0000 7f35d9237640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f35d40d5a80 0x7f35d40b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.470+0000 7f35d9237640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35d40b4b20 0x7f35d41630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.470+0000 7f35d9237640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d40b2760 con 0x7f35d415f400 2024-04-26T20:22:15.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.470+0000 7f35d9237640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f35d40b43e0 con 0x7f35d40b4b20 2024-04-26T20:22:15.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.470+0000 7f35d9237640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f35d40b3ed0 con 0x7f35d40d5a80 2024-04-26T20:22:15.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.471+0000 7f35d359e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35d40b4b20 0x7f35d41630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.471+0000 7f35d359e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35d40b4b20 0x7f35d41630b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40278/0 (socket says 172.21.15.37:40278) 2024-04-26T20:22:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.471+0000 7f35d359e640 1 -- 172.21.15.37:0/3473957001 learned_addr learned my addr 172.21.15.37:0/3473957001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.471+0000 7f35ca59c640 1 --2- 172.21.15.37:0/3473957001 >> v2:172.21.15.134:3300/0 conn(0x7f35d40d5a80 0x7f35d40b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.471+0000 7f35d359e640 1 -- 172.21.15.37:0/3473957001 >> v2:172.21.15.134:3300/0 conn(0x7f35d40d5a80 msgr2=0x7f35d40b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.471+0000 7f35d359e640 1 --2- 172.21.15.37:0/3473957001 >> v2:172.21.15.134:3300/0 conn(0x7f35d40d5a80 0x7f35d40b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.471+0000 7f35d359e640 1 -- 172.21.15.37:0/3473957001 >> v1:172.21.15.134:6789/0 conn(0x7f35d415f400 legacy=0x7f35d40b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.471+0000 7f35d359e640 1 -- 172.21.15.37:0/3473957001 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d40b2a00 con 0x7f35d40b4b20 2024-04-26T20:22:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d359e640 1 --2- 172.21.15.37:0/3473957001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40b4b20 0x7f35d41630b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f35d408ea70 tx=0x7f35cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1fcc7f66d7c073 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d259c640 1 -- 172.21.15.37:0/3473957001 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc036070 con 0x7f35d40b4b20 2024-04-26T20:22:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d259c640 1 -- 172.21.15.37:0/3473957001 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35cc031600 con 0x7f35d40b4b20 2024-04-26T20:22:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d259c640 1 -- 172.21.15.37:0/3473957001 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0319d0 con 0x7f35d40b4b20 2024-04-26T20:22:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d9237640 1 -- 172.21.15.37:0/3473957001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40b4b20 msgr2=0x7f35d41630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d9237640 1 --2- 172.21.15.37:0/3473957001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40b4b20 0x7f35d41630b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f35d408ea70 tx=0x7f35cc02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d9237640 1 -- 172.21.15.37:0/3473957001 shutdown_connections 2024-04-26T20:22:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d9237640 1 --2- 172.21.15.37:0/3473957001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40b4b20 0x7f35d41630b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d9237640 1 --2- 172.21.15.37:0/3473957001 >> v2:172.21.15.134:3300/0 conn(0x7f35d40d5a80 0x7f35d40b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d9237640 1 -- 172.21.15.37:0/3473957001 >> 172.21.15.37:0/3473957001 conn(0x7f35d409e4f0 msgr2=0x7f35d40d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d9237640 1 -- 172.21.15.37:0/3473957001 shutdown_connections 2024-04-26T20:22:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.472+0000 7f35d9237640 1 -- 172.21.15.37:0/3473957001 wait complete. 2024-04-26T20:22:15.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.474+0000 7f35d9237640 1 Processor -- start 2024-04-26T20:22:15.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.474+0000 7f35d9237640 1 -- start start 2024-04-26T20:22:15.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35d9237640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d40b4b20 0x7f35d4159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35d9237640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40d5a80 0x7f35d40ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35d9237640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35d415f400 0x7f35d4160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35d9237640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f35d40b32a0 con 0x7f35d40d5a80 2024-04-26T20:22:15.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35d9237640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35d40b2fc0 con 0x7f35d40b4b20 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35d9237640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f35d40b4f30 con 0x7f35d415f400 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35ca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40d5a80 0x7f35d40ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35ca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40d5a80 0x7f35d40ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40280/0 (socket says 172.21.15.37:40280) 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35ca59c640 1 -- 172.21.15.37:0/2168690231 learned_addr learned my addr 172.21.15.37:0/2168690231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35d359e640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35d415f400 0x7f35d4160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35d2d9d640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d40b4b20 0x7f35d4159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35ca59c640 1 -- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35d415f400 msgr2=0x7f35d4160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35ca59c640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35d415f400 0x7f35d4160e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35ca59c640 1 -- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d40b4b20 msgr2=0x7f35d4159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35ca59c640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d40b4b20 0x7f35d4159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35ca59c640 1 -- 172.21.15.37:0/2168690231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d40ca760 con 0x7f35d40d5a80 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35ca59c640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40d5a80 0x7f35d40ca220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00d5f0 tx=0x7f35bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35cbfff640 1 -- 172.21.15.37:0/2168690231 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc0125f0 con 0x7f35d40d5a80 2024-04-26T20:22:15.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35cbfff640 1 -- 172.21.15.37:0/2168690231 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35bc019070 con 0x7f35d40d5a80 2024-04-26T20:22:15.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35d9237640 1 -- 172.21.15.37:0/2168690231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d40cb120 con 0x7f35d40d5a80 2024-04-26T20:22:15.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35cbfff640 1 -- 172.21.15.37:0/2168690231 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc0129d0 con 0x7f35d40d5a80 2024-04-26T20:22:15.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.475+0000 7f35d9237640 1 -- 172.21.15.37:0/2168690231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d40c9740 con 0x7f35d40d5a80 2024-04-26T20:22:15.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.476+0000 7f35d9237640 1 -- 172.21.15.37:0/2168690231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d406a360 con 0x7f35d40d5a80 2024-04-26T20:22:15.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.477+0000 7f35cbfff640 1 -- 172.21.15.37:0/2168690231 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35bc012b70 con 0x7f35d40d5a80 2024-04-26T20:22:15.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.477+0000 7f35cbfff640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35b406c8f0 0x7f35b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.477+0000 7f35cbfff640 1 -- 172.21.15.37:0/2168690231 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f35bc08bf80 con 0x7f35d40d5a80 2024-04-26T20:22:15.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.478+0000 7f35d2d9d640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35b406c8f0 0x7f35b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.478+0000 7f35d2d9d640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35b406c8f0 0x7f35b406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f35c4000f60 tx=0x7f35c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:15.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.482+0000 7f35cbfff640 1 -- 172.21.15.37:0/2168690231 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f35d40ca760 con 0x7f35d40d5a80 2024-04-26T20:22:15.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.666+0000 7f35d9237640 1 -- 172.21.15.37:0/2168690231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35d40d5270 con 0x7f35d40d5a80 2024-04-26T20:22:15.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.667+0000 7f35cbfff640 1 -- 172.21.15.37:0/2168690231 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f35bc05a3b0 con 0x7f35d40d5a80 2024-04-26T20:22:15.668 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:15.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.668+0000 7f35d9237640 1 -- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35b406c8f0 msgr2=0x7f35b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.669+0000 7f35d9237640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35b406c8f0 0x7f35b406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f35c4000f60 tx=0x7f35c4009040 comp rx=0 tx=0).stop 2024-04-26T20:22:15.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.669+0000 7f35d9237640 1 -- 172.21.15.37:0/2168690231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40d5a80 msgr2=0x7f35d40ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.669+0000 7f35d9237640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40d5a80 0x7f35d40ca220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00d5f0 tx=0x7f35bc00dab0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.669+0000 7f35d9237640 1 -- 172.21.15.37:0/2168690231 shutdown_connections 2024-04-26T20:22:15.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.669+0000 7f35d9237640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35b406c8f0 0x7f35b406edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.669+0000 7f35d9237640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35d415f400 0x7f35d4160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.669+0000 7f35d9237640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40d5a80 0x7f35d40ca220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.669+0000 7f35d9237640 1 --2- 172.21.15.37:0/2168690231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d40b4b20 0x7f35d4159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.669+0000 7f35d9237640 1 -- 172.21.15.37:0/2168690231 >> 172.21.15.37:0/2168690231 conn(0x7f35d409e4f0 msgr2=0x7f35d40d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:15.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.669+0000 7f35d9237640 1 -- 172.21.15.37:0/2168690231 shutdown_connections 2024-04-26T20:22:15.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.669+0000 7f35d9237640 1 -- 172.21.15.37:0/2168690231 wait complete. 2024-04-26T20:22:15.681 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:15.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.783+0000 7fc785825640 1 Processor -- start 2024-04-26T20:22:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.783+0000 7fc785825640 1 -- start start 2024-04-26T20:22:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.784+0000 7fc785825640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc7800b2290 0x7fc7800b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.784+0000 7fc785825640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc7800d26f0 0x7fc7800cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.784+0000 7fc785825640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7800cd640 con 0x7fc7800cf870 2024-04-26T20:22:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.784+0000 7fc785825640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc7800d2b20 con 0x7fc7800b2290 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.784+0000 7fc785825640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc7800cd920 con 0x7fc7800d26f0 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.784+0000 7fc77effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc7800b2290 0x7fc7800b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.784+0000 7fc77effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc7800b2290 0x7fc7800b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60178/0 (socket says 172.21.15.37:60178) 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.784+0000 7fc77effd640 1 -- 172.21.15.37:0/1132213356 learned_addr learned my addr 172.21.15.37:0/1132213356 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.784+0000 7fc77f7fe640 1 --2- 172.21.15.37:0/1132213356 >> v2:172.21.15.134:3300/0 conn(0x7fc7800d26f0 0x7fc7800cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.784+0000 7fc77effd640 1 -- 172.21.15.37:0/1132213356 >> v2:172.21.15.134:3300/0 conn(0x7fc7800d26f0 msgr2=0x7fc7800cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.784+0000 7fc77effd640 1 --2- 172.21.15.37:0/1132213356 >> v2:172.21.15.134:3300/0 conn(0x7fc7800d26f0 0x7fc7800cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.785+0000 7fc77effd640 1 -- 172.21.15.37:0/1132213356 >> v1:172.21.15.134:6789/0 conn(0x7fc7800cf870 legacy=0x7fc7800d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.785+0000 7fc77effd640 1 -- 172.21.15.37:0/1132213356 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7800cdc00 con 0x7fc7800b2290 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.785+0000 7fc77f7fe640 1 --2- 172.21.15.37:0/1132213356 >> v2:172.21.15.134:3300/0 conn(0x7fc7800d26f0 0x7fc7800cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.785+0000 7fc77effd640 1 --2- 172.21.15.37:0/1132213356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7800b2290 0x7fc7800b2930 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc780073b10 tx=0x7fc77802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913f330dd3674c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.785+0000 7fc77dffb640 1 -- 172.21.15.37:0/1132213356 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778037070 con 0x7fc7800b2290 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.785+0000 7fc77dffb640 1 -- 172.21.15.37:0/1132213356 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc778032440 con 0x7fc7800b2290 2024-04-26T20:22:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.785+0000 7fc77dffb640 1 -- 172.21.15.37:0/1132213356 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778032810 con 0x7fc7800b2290 2024-04-26T20:22:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.786+0000 7fc785825640 1 -- 172.21.15.37:0/1132213356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7800b2290 msgr2=0x7fc7800b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.786+0000 7fc785825640 1 --2- 172.21.15.37:0/1132213356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7800b2290 0x7fc7800b2930 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc780073b10 tx=0x7fc77802a230 comp rx=0 tx=0).stop 2024-04-26T20:22:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.786+0000 7fc785825640 1 -- 172.21.15.37:0/1132213356 shutdown_connections 2024-04-26T20:22:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.786+0000 7fc785825640 1 --2- 172.21.15.37:0/1132213356 >> v2:172.21.15.134:3300/0 conn(0x7fc7800d26f0 0x7fc7800cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.786+0000 7fc785825640 1 --2- 172.21.15.37:0/1132213356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7800b2290 0x7fc7800b2930 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.786+0000 7fc785825640 1 -- 172.21.15.37:0/1132213356 >> 172.21.15.37:0/1132213356 conn(0x7fc7800d2ff0 msgr2=0x7fc7800d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.786+0000 7fc785825640 1 -- 172.21.15.37:0/1132213356 shutdown_connections 2024-04-26T20:22:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.786+0000 7fc785825640 1 -- 172.21.15.37:0/1132213356 wait complete. 2024-04-26T20:22:15.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.787+0000 7fc785825640 1 Processor -- start 2024-04-26T20:22:15.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.787+0000 7fc785825640 1 -- start start 2024-04-26T20:22:15.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc785825640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7800b2290 0x7fc780159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc785825640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7800cf870 0x7fc7800b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc785825640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7800d26f0 0x7fc7800b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc785825640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc7800b2670 con 0x7fc7800b2290 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc785825640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7800ce1c0 con 0x7fc7800cf870 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc785825640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc78015ec60 con 0x7fc7800d26f0 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc77effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7800b2290 0x7fc780159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc77effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7800b2290 0x7fc780159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60194/0 (socket says 172.21.15.37:60194) 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc77effd640 1 -- 172.21.15.37:0/306422759 learned_addr learned my addr 172.21.15.37:0/306422759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc77f7fe640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7800d26f0 0x7fc7800b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc77e7fc640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7800cf870 0x7fc7800b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc77effd640 1 -- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7800d26f0 msgr2=0x7fc7800b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc77effd640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7800d26f0 0x7fc7800b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc77effd640 1 -- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7800cf870 msgr2=0x7fc7800b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.788+0000 7fc77effd640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7800cf870 0x7fc7800b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.789+0000 7fc77effd640 1 -- 172.21.15.37:0/306422759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7800b8710 con 0x7fc7800b2290 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.789+0000 7fc77effd640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7800b2290 0x7fc780159810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc778000c00 tx=0x7fc7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.789+0000 7fc767fff640 1 -- 172.21.15.37:0/306422759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778041020 con 0x7fc7800b2290 2024-04-26T20:22:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.789+0000 7fc767fff640 1 -- 172.21.15.37:0/306422759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc778030970 con 0x7fc7800b2290 2024-04-26T20:22:15.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.789+0000 7fc785825640 1 -- 172.21.15.37:0/306422759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7800b89f0 con 0x7fc7800b2290 2024-04-26T20:22:15.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.789+0000 7fc767fff640 1 -- 172.21.15.37:0/306422759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778037040 con 0x7fc7800b2290 2024-04-26T20:22:15.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.789+0000 7fc785825640 1 -- 172.21.15.37:0/306422759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7800ccd60 con 0x7fc7800b2290 2024-04-26T20:22:15.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.790+0000 7fc785825640 1 -- 172.21.15.37:0/306422759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78006a360 con 0x7fc7800b2290 2024-04-26T20:22:15.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.790+0000 7fc767fff640 1 -- 172.21.15.37:0/306422759 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc778044020 con 0x7fc7800b2290 2024-04-26T20:22:15.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.791+0000 7fc767fff640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc76c06c920 0x7fc76c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.791+0000 7fc767fff640 1 -- 172.21.15.37:0/306422759 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc7780ac450 con 0x7fc7800b2290 2024-04-26T20:22:15.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.791+0000 7fc77e7fc640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc76c06c920 0x7fc76c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.792+0000 7fc77e7fc640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc76c06c920 0x7fc76c06ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7800cc840 tx=0x7fc774007e30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:15.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.795+0000 7fc767fff640 1 -- 172.21.15.37:0/306422759 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc77804b030 con 0x7fc7800b2290 2024-04-26T20:22:15.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.980+0000 7fc785825640 1 -- 172.21.15.37:0/306422759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc7800b8bc0 con 0x7fc7800b2290 2024-04-26T20:22:15.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.980+0000 7fc767fff640 1 -- 172.21.15.37:0/306422759 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc77807a230 con 0x7fc7800b2290 2024-04-26T20:22:15.981 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:15.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 -- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc76c06c920 msgr2=0x7fc76c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc76c06c920 0x7fc76c06ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7800cc840 tx=0x7fc774007e30 comp rx=0 tx=0).stop 2024-04-26T20:22:15.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 -- 172.21.15.37:0/306422759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7800b2290 msgr2=0x7fc780159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7800b2290 0x7fc780159810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc778000c00 tx=0x7fc7780046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 -- 172.21.15.37:0/306422759 shutdown_connections 2024-04-26T20:22:15.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc76c06c920 0x7fc76c06ede0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7800d26f0 0x7fc7800b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7800cf870 0x7fc7800b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 --2- 172.21.15.37:0/306422759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7800b2290 0x7fc780159810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 -- 172.21.15.37:0/306422759 >> 172.21.15.37:0/306422759 conn(0x7fc7800d2ff0 msgr2=0x7fc780070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:15.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 -- 172.21.15.37:0/306422759 shutdown_connections 2024-04-26T20:22:15.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:15.982+0000 7fc785825640 1 -- 172.21.15.37:0/306422759 wait complete. 2024-04-26T20:22:15.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:16.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.096+0000 7f095c3f0640 1 Processor -- start 2024-04-26T20:22:16.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.097+0000 7f095c3f0640 1 -- start start 2024-04-26T20:22:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.097+0000 7f095c3f0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f09540b5510 0x7f09540d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.097+0000 7f095c3f0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09540b58f0 0x7f09540d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.097+0000 7f095c3f0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09540c9cc0 con 0x7f09540b6820 2024-04-26T20:22:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.097+0000 7f095c3f0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f09540b5d80 con 0x7f09540b58f0 2024-04-26T20:22:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.097+0000 7f095c3f0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f09540b5fd0 con 0x7f09540b5510 2024-04-26T20:22:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.098+0000 7f095a98d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09540b58f0 0x7f09540d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.098+0000 7f095a98d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09540b58f0 0x7f09540d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60210/0 (socket says 172.21.15.37:60210) 2024-04-26T20:22:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.098+0000 7f095a98d640 1 -- 172.21.15.37:0/3895954679 learned_addr learned my addr 172.21.15.37:0/3895954679 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.098+0000 7f095a18c640 1 --2- 172.21.15.37:0/3895954679 >> v2:172.21.15.134:3300/0 conn(0x7f09540b5510 0x7f09540d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.098+0000 7f095a98d640 1 -- 172.21.15.37:0/3895954679 >> v2:172.21.15.134:3300/0 conn(0x7f09540b5510 msgr2=0x7f09540d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.098+0000 7f095a98d640 1 --2- 172.21.15.37:0/3895954679 >> v2:172.21.15.134:3300/0 conn(0x7f09540b5510 0x7f09540d1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.098+0000 7f095a98d640 1 -- 172.21.15.37:0/3895954679 >> v1:172.21.15.134:6789/0 conn(0x7f09540b6820 legacy=0x7f09540d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.098+0000 7f095a98d640 1 -- 172.21.15.37:0/3895954679 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09540d5c80 con 0x7f09540b58f0 2024-04-26T20:22:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.098+0000 7f095a18c640 1 --2- 172.21.15.37:0/3895954679 >> v2:172.21.15.134:3300/0 conn(0x7f09540b5510 0x7f09540d1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095a98d640 1 --2- 172.21.15.37:0/3895954679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09540b58f0 0x7f09540d1dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f09540b5f00 tx=0x7f094c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7660454f7b82d01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095918a640 1 -- 172.21.15.37:0/3895954679 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c036070 con 0x7f09540b58f0 2024-04-26T20:22:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095918a640 1 -- 172.21.15.37:0/3895954679 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094c031600 con 0x7f09540b58f0 2024-04-26T20:22:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095918a640 1 -- 172.21.15.37:0/3895954679 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0319d0 con 0x7f09540b58f0 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095c3f0640 1 -- 172.21.15.37:0/3895954679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09540b58f0 msgr2=0x7f09540d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095c3f0640 1 --2- 172.21.15.37:0/3895954679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09540b58f0 0x7f09540d1dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f09540b5f00 tx=0x7f094c02a640 comp rx=0 tx=0).stop 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095c3f0640 1 -- 172.21.15.37:0/3895954679 shutdown_connections 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095c3f0640 1 --2- 172.21.15.37:0/3895954679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09540b58f0 0x7f09540d1dd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095c3f0640 1 --2- 172.21.15.37:0/3895954679 >> v2:172.21.15.134:3300/0 conn(0x7f09540b5510 0x7f09540d1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095c3f0640 1 -- 172.21.15.37:0/3895954679 >> 172.21.15.37:0/3895954679 conn(0x7f09540d5160 msgr2=0x7f09540d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095c3f0640 1 -- 172.21.15.37:0/3895954679 shutdown_connections 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.099+0000 7f095c3f0640 1 -- 172.21.15.37:0/3895954679 wait complete. 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.101+0000 7f095c3f0640 1 Processor -- start 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.101+0000 7f095c3f0640 1 -- start start 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.101+0000 7f095c3f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09540b5510 0x7f09540cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.101+0000 7f095c3f0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09540b58f0 0x7f09540ba750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.101+0000 7f095c3f0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09540b6820 0x7f09541544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.101+0000 7f095c3f0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09540d0b60 con 0x7f09540b5510 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.101+0000 7f095c3f0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09540d0880 con 0x7f09540b58f0 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.101+0000 7f095c3f0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09540d4100 con 0x7f09540b6820 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.101+0000 7f095a18c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09540b5510 0x7f09540cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.101+0000 7f095a18c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09540b5510 0x7f09540cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60216/0 (socket says 172.21.15.37:60216) 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.101+0000 7f095a18c640 1 -- 172.21.15.37:0/3054443240 learned_addr learned my addr 172.21.15.37:0/3054443240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f095a98d640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09540b6820 0x7f09541544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f095998b640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09540b58f0 0x7f09540ba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f095a18c640 1 -- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09540b6820 msgr2=0x7f09541544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f095a18c640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09540b6820 0x7f09541544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f095a18c640 1 -- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09540b58f0 msgr2=0x7f09540ba750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f095a18c640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09540b58f0 0x7f09540ba750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f095a18c640 1 -- 172.21.15.37:0/3054443240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09540bac90 con 0x7f09540b5510 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f095998b640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09540b58f0 0x7f09540ba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f095a18c640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09540b5510 0x7f09540cc2c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f095000c7f0 tx=0x7f095000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f094affd640 1 -- 172.21.15.37:0/3054443240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950015070 con 0x7f09540b5510 2024-04-26T20:22:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f094affd640 1 -- 172.21.15.37:0/3054443240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09500079d0 con 0x7f09540b5510 2024-04-26T20:22:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f095c3f0640 1 -- 172.21.15.37:0/3054443240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09540ce960 con 0x7f09540b5510 2024-04-26T20:22:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f094affd640 1 -- 172.21.15.37:0/3054443240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950007da0 con 0x7f09540b5510 2024-04-26T20:22:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.102+0000 7f095c3f0640 1 -- 172.21.15.37:0/3054443240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09540cef20 con 0x7f09540b5510 2024-04-26T20:22:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.103+0000 7f095c3f0640 1 -- 172.21.15.37:0/3054443240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09540693e0 con 0x7f09540b5510 2024-04-26T20:22:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.103+0000 7f094affd640 1 -- 172.21.15.37:0/3054443240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0950018070 con 0x7f09540b5510 2024-04-26T20:22:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.104+0000 7f094affd640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f093806c8f0 0x7f093806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.104+0000 7f094affd640 1 -- 172.21.15.37:0/3054443240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f095008b9e0 con 0x7f09540b5510 2024-04-26T20:22:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.104+0000 7f095998b640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f093806c8f0 0x7f093806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.105+0000 7f095998b640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f093806c8f0 0x7f093806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0944006fd0 tx=0x7f0944008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:16.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.108+0000 7f094affd640 1 -- 172.21.15.37:0/3054443240 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f09500597c0 con 0x7f09540b5510 2024-04-26T20:22:16.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.297+0000 7f095c3f0640 1 -- 172.21.15.37:0/3054443240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f09540d57e0 con 0x7f09540b5510 2024-04-26T20:22:16.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.297+0000 7f094affd640 1 -- 172.21.15.37:0/3054443240 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f095005dac0 con 0x7f09540b5510 2024-04-26T20:22:16.299 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:16.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 -- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f093806c8f0 msgr2=0x7f093806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f093806c8f0 0x7f093806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0944006fd0 tx=0x7f0944008040 comp rx=0 tx=0).stop 2024-04-26T20:22:16.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 -- 172.21.15.37:0/3054443240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09540b5510 msgr2=0x7f09540cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09540b5510 0x7f09540cc2c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f095000c7f0 tx=0x7f095000cbf0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 -- 172.21.15.37:0/3054443240 shutdown_connections 2024-04-26T20:22:16.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f093806c8f0 0x7f093806edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09540b6820 0x7f09541544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09540b58f0 0x7f09540ba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 --2- 172.21.15.37:0/3054443240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09540b5510 0x7f09540cc2c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 -- 172.21.15.37:0/3054443240 >> 172.21.15.37:0/3054443240 conn(0x7f09540d5160 msgr2=0x7f09540b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:16.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 -- 172.21.15.37:0/3054443240 shutdown_connections 2024-04-26T20:22:16.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.299+0000 7f095c3f0640 1 -- 172.21.15.37:0/3054443240 wait complete. 2024-04-26T20:22:16.312 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:16.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.411+0000 7fbc6a6d1640 1 Processor -- start 2024-04-26T20:22:16.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.411+0000 7fbc6a6d1640 1 -- start start 2024-04-26T20:22:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.412+0000 7fbc6a6d1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbc6415f760 0x7fbc640b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.412+0000 7fbc6a6d1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc640d3de0 con 0x7fbc6415f380 2024-04-26T20:22:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.412+0000 7fbc6a6d1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc640d40c0 con 0x7fbc640d22e0 2024-04-26T20:22:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.412+0000 7fbc6a6d1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fbc640d43a0 con 0x7fbc6415f760 2024-04-26T20:22:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.412+0000 7fbc68c6e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbc6415f760 0x7fbc640b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.412+0000 7fbc63fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbc6415f380 0x7fbc640d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32838/0 (socket says 172.21.15.37:32838) 2024-04-26T20:22:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.412+0000 7fbc63fff640 1 -- 172.21.15.37:0/1840759080 learned_addr learned my addr 172.21.15.37:0/1840759080 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.412+0000 7fbc68c6e640 1 -- 172.21.15.37:0/1840759080 >> v1:172.21.15.134:6789/0 conn(0x7fbc640d22e0 legacy=0x7fbc640b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.412+0000 7fbc68c6e640 1 -- 172.21.15.37:0/1840759080 >> v1:172.21.15.37:6789/0 conn(0x7fbc6415f380 legacy=0x7fbc640d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.412+0000 7fbc68c6e640 1 -- 172.21.15.37:0/1840759080 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc640b8280 con 0x7fbc6415f760 2024-04-26T20:22:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.412+0000 7fbc62ffd640 1 -- 172.21.15.37:0/1840759080 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1500536820 0 0) 0x7fbc640d3de0 con 0x7fbc6415f380 2024-04-26T20:22:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.413+0000 7fbc68c6e640 1 --2- 172.21.15.37:0/1840759080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc6415f760 0x7fbc640b9190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbc640d4520 tx=0x7fbc5c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04ed687ce945d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.413+0000 7fbc62ffd640 1 -- 172.21.15.37:0/1840759080 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c007740 con 0x7fbc6415f760 2024-04-26T20:22:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.413+0000 7fbc62ffd640 1 -- 172.21.15.37:0/1840759080 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5c0078e0 con 0x7fbc6415f760 2024-04-26T20:22:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.413+0000 7fbc62ffd640 1 -- 172.21.15.37:0/1840759080 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c007cb0 con 0x7fbc6415f760 2024-04-26T20:22:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.414+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1840759080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc6415f760 msgr2=0x7fbc640b9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.414+0000 7fbc6a6d1640 1 --2- 172.21.15.37:0/1840759080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc6415f760 0x7fbc640b9190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbc640d4520 tx=0x7fbc5c02a750 comp rx=0 tx=0).stop 2024-04-26T20:22:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.414+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1840759080 shutdown_connections 2024-04-26T20:22:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.414+0000 7fbc6a6d1640 1 --2- 172.21.15.37:0/1840759080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc6415f760 0x7fbc640b9190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.414+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1840759080 >> 172.21.15.37:0/1840759080 conn(0x7fbc6409e560 msgr2=0x7fbc640d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.414+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1840759080 shutdown_connections 2024-04-26T20:22:16.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.414+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1840759080 wait complete. 2024-04-26T20:22:16.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.415+0000 7fbc6a6d1640 1 Processor -- start 2024-04-26T20:22:16.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.415+0000 7fbc6a6d1640 1 -- start start 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc6a6d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc640d22e0 0x7fbc6415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc6a6d1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc6415f380 0x7fbc64157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc6a6d1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc6415f760 0x7fbc64157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc6a6d1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbc640d1280 con 0x7fbc640d22e0 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc6a6d1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc640d1100 con 0x7fbc6415f380 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc6a6d1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbc640d5b50 con 0x7fbc6415f760 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc63fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc640d22e0 0x7fbc6415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc63fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc640d22e0 0x7fbc6415adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60234/0 (socket says 172.21.15.37:60234) 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc63fff640 1 -- 172.21.15.37:0/1721117165 learned_addr learned my addr 172.21.15.37:0/1721117165 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc637fe640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc6415f380 0x7fbc64157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc68c6e640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc6415f760 0x7fbc64157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc63fff640 1 -- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc6415f760 msgr2=0x7fbc64157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc63fff640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc6415f760 0x7fbc64157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc63fff640 1 -- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc6415f380 msgr2=0x7fbc64157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc63fff640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc6415f380 0x7fbc64157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc63fff640 1 -- 172.21.15.37:0/1721117165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc640ce710 con 0x7fbc640d22e0 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.416+0000 7fbc63fff640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc640d22e0 0x7fbc6415adc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00c4d0 tx=0x7fbc4c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.417+0000 7fbc60ff9640 1 -- 172.21.15.37:0/1721117165 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c018070 con 0x7fbc640d22e0 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.417+0000 7fbc60ff9640 1 -- 172.21.15.37:0/1721117165 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c007500 con 0x7fbc640d22e0 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.417+0000 7fbc60ff9640 1 -- 172.21.15.37:0/1721117165 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c004a20 con 0x7fbc640d22e0 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.417+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1721117165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc640ce9f0 con 0x7fbc640d22e0 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.417+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1721117165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc640cefb0 con 0x7fbc640d22e0 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.418+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1721117165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6406a360 con 0x7fbc640d22e0 2024-04-26T20:22:16.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.418+0000 7fbc60ff9640 1 -- 172.21.15.37:0/1721117165 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc4c01b070 con 0x7fbc640d22e0 2024-04-26T20:22:16.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.418+0000 7fbc60ff9640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc4406c920 0x7fbc4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.419+0000 7fbc60ff9640 1 -- 172.21.15.37:0/1721117165 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbc4c08ae60 con 0x7fbc640d22e0 2024-04-26T20:22:16.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.419+0000 7fbc637fe640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc4406c920 0x7fbc4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.419+0000 7fbc637fe640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc4406c920 0x7fbc4406ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbc64155a30 tx=0x7fbc54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:16.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.423+0000 7fbc60ff9640 1 -- 172.21.15.37:0/1721117165 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc4c059320 con 0x7fbc640d22e0 2024-04-26T20:22:16.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.608+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1721117165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc6415ece0 con 0x7fbc640d22e0 2024-04-26T20:22:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.609+0000 7fbc60ff9640 1 -- 172.21.15.37:0/1721117165 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7fbc4c058c20 con 0x7fbc640d22e0 2024-04-26T20:22:16.610 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:16.610 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc4406c920 msgr2=0x7fbc4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc4406c920 0x7fbc4406ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbc64155a30 tx=0x7fbc54009040 comp rx=0 tx=0).stop 2024-04-26T20:22:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1721117165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc640d22e0 msgr2=0x7fbc6415adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc640d22e0 0x7fbc6415adc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00c4d0 tx=0x7fbc4c00c990 comp rx=0 tx=0).stop 2024-04-26T20:22:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1721117165 shutdown_connections 2024-04-26T20:22:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc4406c920 0x7fbc4406ede0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc6415f760 0x7fbc64157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc6415f380 0x7fbc64157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 --2- 172.21.15.37:0/1721117165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc640d22e0 0x7fbc6415adc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1721117165 >> 172.21.15.37:0/1721117165 conn(0x7fbc6409e560 msgr2=0x7fbc640b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1721117165 shutdown_connections 2024-04-26T20:22:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.611+0000 7fbc6a6d1640 1 -- 172.21.15.37:0/1721117165 wait complete. 2024-04-26T20:22:16.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:16.624 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:16.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.726+0000 7f6b1b4a9640 1 Processor -- start 2024-04-26T20:22:16.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.726+0000 7f6b1b4a9640 1 -- start start 2024-04-26T20:22:16.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b1b4a9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6b1415f470 0x7f6b140d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b1b4a9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6b140d0d30 0x7f6b140d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b1b4a9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b140d2ed0 con 0x7f6b140d1fe0 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b1b4a9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6b140d1190 con 0x7f6b1415f470 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b1b4a9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6b140d31b0 con 0x7f6b140d0d30 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b19245640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6b1415f470 0x7f6b140d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b19a46640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6b140d1fe0 0x7f6b14162fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32846/0 (socket says 172.21.15.37:32846) 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b19245640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6b1415f470 0x7f6b140d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60250/0 (socket says 172.21.15.37:60250) 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b19a46640 1 -- 172.21.15.37:0/4245334076 learned_addr learned my addr 172.21.15.37:0/4245334076 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b18a44640 1 --2- 172.21.15.37:0/4245334076 >> v2:172.21.15.134:3300/0 conn(0x7f6b140d0d30 0x7f6b140d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b19245640 1 -- 172.21.15.37:0/4245334076 >> v2:172.21.15.134:3300/0 conn(0x7f6b140d0d30 msgr2=0x7f6b140d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b19245640 1 --2- 172.21.15.37:0/4245334076 >> v2:172.21.15.134:3300/0 conn(0x7f6b140d0d30 0x7f6b140d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b19245640 1 -- 172.21.15.37:0/4245334076 >> v1:172.21.15.37:6789/0 conn(0x7f6b140d1fe0 legacy=0x7f6b14162fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b0bfff640 1 -- 172.21.15.37:0/4245334076 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3378057262 0 0) 0x7f6b140d2ed0 con 0x7f6b140d1fe0 2024-04-26T20:22:16.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.727+0000 7f6b19245640 1 -- 172.21.15.37:0/4245334076 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b140d3490 con 0x7f6b1415f470 2024-04-26T20:22:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.728+0000 7f6b19245640 1 --2- 172.21.15.37:0/4245334076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1415f470 0x7f6b140d07f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b140bb210 tx=0x7f6b1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad6a4d9b869afcc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.728+0000 7f6b0bfff640 1 -- 172.21.15.37:0/4245334076 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10037070 con 0x7f6b1415f470 2024-04-26T20:22:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.728+0000 7f6b0bfff640 1 -- 172.21.15.37:0/4245334076 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b10032440 con 0x7f6b1415f470 2024-04-26T20:22:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.728+0000 7f6b0bfff640 1 -- 172.21.15.37:0/4245334076 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10032810 con 0x7f6b1415f470 2024-04-26T20:22:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.729+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/4245334076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1415f470 msgr2=0x7f6b140d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.729+0000 7f6b1b4a9640 1 --2- 172.21.15.37:0/4245334076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1415f470 0x7f6b140d07f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b140bb210 tx=0x7f6b1002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.729+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/4245334076 shutdown_connections 2024-04-26T20:22:16.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.729+0000 7f6b1b4a9640 1 --2- 172.21.15.37:0/4245334076 >> v2:172.21.15.134:3300/0 conn(0x7f6b140d0d30 0x7f6b140d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.729+0000 7f6b1b4a9640 1 --2- 172.21.15.37:0/4245334076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1415f470 0x7f6b140d07f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.729+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/4245334076 >> 172.21.15.37:0/4245334076 conn(0x7f6b1409e510 msgr2=0x7f6b140d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:16.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.729+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/4245334076 shutdown_connections 2024-04-26T20:22:16.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.729+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/4245334076 wait complete. 2024-04-26T20:22:16.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.730+0000 7f6b1b4a9640 1 Processor -- start 2024-04-26T20:22:16.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.730+0000 7f6b1b4a9640 1 -- start start 2024-04-26T20:22:16.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b1b4a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b140d0d30 0x7f6b1414b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b1b4a9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b140d1fe0 0x7f6b140c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b1b4a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1415f470 0x7f6b140c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b1b4a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1415f850 con 0x7f6b1415f470 2024-04-26T20:22:16.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b1b4a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b140d3a50 con 0x7f6b140d0d30 2024-04-26T20:22:16.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b1b4a9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6b140d3f70 con 0x7f6b140d1fe0 2024-04-26T20:22:16.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b19a46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1415f470 0x7f6b140c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b19a46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1415f470 0x7f6b140c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60262/0 (socket says 172.21.15.37:60262) 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b19a46640 1 -- 172.21.15.37:0/3339307473 learned_addr learned my addr 172.21.15.37:0/3339307473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b19245640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b140d0d30 0x7f6b1414b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b18a44640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b140d1fe0 0x7f6b140c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b19a46640 1 -- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b140d1fe0 msgr2=0x7f6b140c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b19a46640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b140d1fe0 0x7f6b140c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b19a46640 1 -- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b140d0d30 msgr2=0x7f6b1414b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b19a46640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b140d0d30 0x7f6b1414b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b19a46640 1 -- 172.21.15.37:0/3339307473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b140cea50 con 0x7f6b1415f470 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b19245640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b140d0d30 0x7f6b1414b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.731+0000 7f6b19a46640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1415f470 0x7f6b140c91d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00c700 tx=0x7f6b0c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.732+0000 7f6b09ffb640 1 -- 172.21.15.37:0/3339307473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c018070 con 0x7f6b1415f470 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.732+0000 7f6b09ffb640 1 -- 172.21.15.37:0/3339307473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b0c00ce10 con 0x7f6b1415f470 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.732+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/3339307473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b140b8a20 con 0x7f6b1415f470 2024-04-26T20:22:16.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.732+0000 7f6b09ffb640 1 -- 172.21.15.37:0/3339307473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00a880 con 0x7f6b1415f470 2024-04-26T20:22:16.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.732+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/3339307473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b140cdd00 con 0x7f6b1415f470 2024-04-26T20:22:16.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.733+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/3339307473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1406a360 con 0x7f6b1415f470 2024-04-26T20:22:16.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.733+0000 7f6b09ffb640 1 -- 172.21.15.37:0/3339307473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b0c01b070 con 0x7f6b1415f470 2024-04-26T20:22:16.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.734+0000 7f6b09ffb640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6af806c8f0 0x7f6af806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.734+0000 7f6b09ffb640 1 -- 172.21.15.37:0/3339307473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6b0c08b040 con 0x7f6b1415f470 2024-04-26T20:22:16.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.734+0000 7f6b19245640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6af806c8f0 0x7f6af806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.735+0000 7f6b19245640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6af806c8f0 0x7f6af806edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6b10034000 tx=0x7f6b1002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:16.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.738+0000 7f6b09ffb640 1 -- 172.21.15.37:0/3339307473 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b0c059500 con 0x7f6b1415f470 2024-04-26T20:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.926+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/3339307473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6b140d5870 con 0x7f6b1415f470 2024-04-26T20:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.927+0000 7f6b09ffb640 1 -- 172.21.15.37:0/3339307473 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f6b0c058e00 con 0x7f6b1415f470 2024-04-26T20:22:16.928 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:16.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6af806c8f0 msgr2=0x7f6af806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6af806c8f0 0x7f6af806edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6b10034000 tx=0x7f6b1002e000 comp rx=0 tx=0).stop 2024-04-26T20:22:16.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/3339307473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1415f470 msgr2=0x7f6b140c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:16.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1415f470 0x7f6b140c91d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00c700 tx=0x7f6b0c00cbc0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/3339307473 shutdown_connections 2024-04-26T20:22:16.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1415f470 0x7f6b140c91d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6af806c8f0 0x7f6af806edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b140d1fe0 0x7f6b140c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 --2- 172.21.15.37:0/3339307473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b140d0d30 0x7f6b1414b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/3339307473 >> 172.21.15.37:0/3339307473 conn(0x7f6b1409e510 msgr2=0x7f6b1415e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:16.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/3339307473 shutdown_connections 2024-04-26T20:22:16.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:16.929+0000 7f6b1b4a9640 1 -- 172.21.15.37:0/3339307473 wait complete. 2024-04-26T20:22:16.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:17.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.044+0000 7f198fcea640 1 Processor -- start 2024-04-26T20:22:17.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.044+0000 7f198fcea640 1 -- start start 2024-04-26T20:22:17.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198fcea640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f198815f9c0 0x7f19880d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198fcea640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f19880cf990 0x7f19880cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198fcea640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19880b2550 con 0x7f1988163380 2024-04-26T20:22:17.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198fcea640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f19880d0340 con 0x7f19880cf990 2024-04-26T20:22:17.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198fcea640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f19880b3760 con 0x7f198815f9c0 2024-04-26T20:22:17.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198d285640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f19880cf990 0x7f19880cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198d285640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f19880cf990 0x7f19880cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60278/0 (socket says 172.21.15.37:60278) 2024-04-26T20:22:17.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198d285640 1 -- 172.21.15.37:0/2544460840 learned_addr learned my addr 172.21.15.37:0/2544460840 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:17.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198da86640 1 --2- 172.21.15.37:0/2544460840 >> v2:172.21.15.134:3300/0 conn(0x7f198815f9c0 0x7f19880d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198d285640 1 -- 172.21.15.37:0/2544460840 >> v2:172.21.15.134:3300/0 conn(0x7f198815f9c0 msgr2=0x7f19880d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:17.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198d285640 1 --2- 172.21.15.37:0/2544460840 >> v2:172.21.15.134:3300/0 conn(0x7f198815f9c0 0x7f19880d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198d285640 1 -- 172.21.15.37:0/2544460840 >> v1:172.21.15.134:6789/0 conn(0x7f1988163380 legacy=0x7f19881637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198d285640 1 -- 172.21.15.37:0/2544460840 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198806dcf0 con 0x7f19880cf990 2024-04-26T20:22:17.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.045+0000 7f198da86640 1 --2- 172.21.15.37:0/2544460840 >> v2:172.21.15.134:3300/0 conn(0x7f198815f9c0 0x7f19880d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:17.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.046+0000 7f198d285640 1 --2- 172.21.15.37:0/2544460840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19880cf990 0x7f19880cfe00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19880d2ee0 tx=0x7f197802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfe947e0795c297 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:17.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.046+0000 7f198ca84640 1 -- 172.21.15.37:0/2544460840 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978037070 con 0x7f19880cf990 2024-04-26T20:22:17.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.046+0000 7f198ca84640 1 -- 172.21.15.37:0/2544460840 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1978032440 con 0x7f19880cf990 2024-04-26T20:22:17.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.046+0000 7f198ca84640 1 -- 172.21.15.37:0/2544460840 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978032810 con 0x7f19880cf990 2024-04-26T20:22:17.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.046+0000 7f198fcea640 1 -- 172.21.15.37:0/2544460840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19880cf990 msgr2=0x7f19880cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.046+0000 7f198fcea640 1 --2- 172.21.15.37:0/2544460840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19880cf990 0x7f19880cfe00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19880d2ee0 tx=0x7f197802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.046+0000 7f198fcea640 1 -- 172.21.15.37:0/2544460840 shutdown_connections 2024-04-26T20:22:17.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.046+0000 7f198fcea640 1 --2- 172.21.15.37:0/2544460840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19880cf990 0x7f19880cfe00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.046+0000 7f198fcea640 1 --2- 172.21.15.37:0/2544460840 >> v2:172.21.15.134:3300/0 conn(0x7f198815f9c0 0x7f19880d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.047+0000 7f198fcea640 1 -- 172.21.15.37:0/2544460840 >> 172.21.15.37:0/2544460840 conn(0x7f19880d2f80 msgr2=0x7f19880d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:17.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.047+0000 7f198fcea640 1 -- 172.21.15.37:0/2544460840 shutdown_connections 2024-04-26T20:22:17.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.047+0000 7f198fcea640 1 -- 172.21.15.37:0/2544460840 wait complete. 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.048+0000 7f198fcea640 1 Processor -- start 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.048+0000 7f198fcea640 1 -- start start 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.048+0000 7f198fcea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19880cf990 0x7f198814c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.048+0000 7f198fcea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198815f9c0 0x7f19880c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.048+0000 7f198fcea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1988163380 0x7f19880c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.048+0000 7f198fcea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f198806e590 con 0x7f198815f9c0 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.048+0000 7f198fcea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f198806e2b0 con 0x7f19880cf990 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.048+0000 7f198fcea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f198806e870 con 0x7f1988163380 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198d285640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198815f9c0 0x7f19880c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198d285640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198815f9c0 0x7f19880c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60294/0 (socket says 172.21.15.37:60294) 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198d285640 1 -- 172.21.15.37:0/616785132 learned_addr learned my addr 172.21.15.37:0/616785132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198da86640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19880cf990 0x7f198814c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198d285640 1 -- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1988163380 msgr2=0x7f19880c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198e287640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1988163380 0x7f19880c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198d285640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1988163380 0x7f19880c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198d285640 1 -- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19880cf990 msgr2=0x7f198814c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198d285640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19880cf990 0x7f198814c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198d285640 1 -- 172.21.15.37:0/616785132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19881565b0 con 0x7f198815f9c0 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198e287640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1988163380 0x7f19880c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198d285640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198815f9c0 0x7f19880c8240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f197802a220 tx=0x7f19780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f197e7fc640 1 -- 172.21.15.37:0/616785132 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978041020 con 0x7f198815f9c0 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f197e7fc640 1 -- 172.21.15.37:0/616785132 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19780306c0 con 0x7f198815f9c0 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198fcea640 1 -- 172.21.15.37:0/616785132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1988160e20 con 0x7f198815f9c0 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f197e7fc640 1 -- 172.21.15.37:0/616785132 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978037040 con 0x7f198815f9c0 2024-04-26T20:22:17.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.049+0000 7f198fcea640 1 -- 172.21.15.37:0/616785132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19881613e0 con 0x7f198815f9c0 2024-04-26T20:22:17.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.050+0000 7f198fcea640 1 -- 172.21.15.37:0/616785132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198806a360 con 0x7f198815f9c0 2024-04-26T20:22:17.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.051+0000 7f197e7fc640 1 -- 172.21.15.37:0/616785132 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1978044020 con 0x7f198815f9c0 2024-04-26T20:22:17.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.051+0000 7f197e7fc640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f196c06c8f0 0x7f196c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.051+0000 7f197e7fc640 1 -- 172.21.15.37:0/616785132 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f19780ac460 con 0x7f198815f9c0 2024-04-26T20:22:17.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.051+0000 7f198da86640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f196c06c8f0 0x7f196c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.052+0000 7f198da86640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f196c06c8f0 0x7f196c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1984006fd0 tx=0x7f1984008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:17.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.055+0000 7f197e7fc640 1 -- 172.21.15.37:0/616785132 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f197804b030 con 0x7f198815f9c0 2024-04-26T20:22:17.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.240+0000 7f198fcea640 1 -- 172.21.15.37:0/616785132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19880cfd70 con 0x7f198815f9c0 2024-04-26T20:22:17.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.240+0000 7f197e7fc640 1 -- 172.21.15.37:0/616785132 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f197807a800 con 0x7f198815f9c0 2024-04-26T20:22:17.242 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 -- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f196c06c8f0 msgr2=0x7f196c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f196c06c8f0 0x7f196c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1984006fd0 tx=0x7f1984008040 comp rx=0 tx=0).stop 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 -- 172.21.15.37:0/616785132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198815f9c0 msgr2=0x7f19880c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198815f9c0 0x7f19880c8240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f197802a220 tx=0x7f19780046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 -- 172.21.15.37:0/616785132 shutdown_connections 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f196c06c8f0 0x7f196c06edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1988163380 0x7f19880c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198815f9c0 0x7f19880c8240 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 --2- 172.21.15.37:0/616785132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19880cf990 0x7f198814c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 -- 172.21.15.37:0/616785132 >> 172.21.15.37:0/616785132 conn(0x7f19880d2f80 msgr2=0x7f19880afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 -- 172.21.15.37:0/616785132 shutdown_connections 2024-04-26T20:22:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.242+0000 7f198fcea640 1 -- 172.21.15.37:0/616785132 wait complete. 2024-04-26T20:22:17.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.357+0000 7ff9e27c9640 1 Processor -- start 2024-04-26T20:22:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.357+0000 7ff9e27c9640 1 -- start start 2024-04-26T20:22:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9e27c9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff9dc0b2290 0x7ff9dc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9e27c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0d26f0 0x7ff9dc0cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9e27c9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9dc0cd640 con 0x7ff9dc0cf870 2024-04-26T20:22:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9e27c9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff9dc0cd920 con 0x7ff9dc0b2290 2024-04-26T20:22:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9e27c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc0cdc00 con 0x7ff9dc0d26f0 2024-04-26T20:22:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9dbfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff9dc0b2290 0x7ff9dc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9db7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff9dc0cf870 0x7ff9dc0d1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32852/0 (socket says 172.21.15.37:32852) 2024-04-26T20:22:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9db7fe640 1 -- 172.21.15.37:0/2759038956 learned_addr learned my addr 172.21.15.37:0/2759038956 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9e0d66640 1 --2- 172.21.15.37:0/2759038956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0d26f0 0x7ff9dc0cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9daffd640 1 -- 172.21.15.37:0/2759038956 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3286890256 0 0) 0x7ff9dc0cd640 con 0x7ff9dc0cf870 2024-04-26T20:22:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9daffd640 1 -- 172.21.15.37:0/2759038956 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9c8003520 con 0x7ff9dc0cf870 2024-04-26T20:22:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9dbfff640 1 -- 172.21.15.37:0/2759038956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0d26f0 msgr2=0x7ff9dc0cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9dbfff640 1 --2- 172.21.15.37:0/2759038956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0d26f0 0x7ff9dc0cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9dbfff640 1 -- 172.21.15.37:0/2759038956 >> v1:172.21.15.37:6789/0 conn(0x7ff9dc0cf870 legacy=0x7ff9dc0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9dbfff640 1 -- 172.21.15.37:0/2759038956 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc0cdee0 con 0x7ff9dc0b2290 2024-04-26T20:22:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.358+0000 7ff9e0d66640 1 --2- 172.21.15.37:0/2759038956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0d26f0 0x7ff9dc0cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.359+0000 7ff9dbfff640 1 --2- 172.21.15.37:0/2759038956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9dc0b2290 0x7ff9dc0b2930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0097a0 tx=0x7ff9cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc67d15a3663fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.359+0000 7ff9daffd640 1 -- 172.21.15.37:0/2759038956 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc037070 con 0x7ff9dc0b2290 2024-04-26T20:22:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.359+0000 7ff9daffd640 1 -- 172.21.15.37:0/2759038956 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9cc032440 con 0x7ff9dc0b2290 2024-04-26T20:22:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.359+0000 7ff9daffd640 1 -- 172.21.15.37:0/2759038956 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc032810 con 0x7ff9dc0b2290 2024-04-26T20:22:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.360+0000 7ff9e27c9640 1 -- 172.21.15.37:0/2759038956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9dc0b2290 msgr2=0x7ff9dc0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.360+0000 7ff9e27c9640 1 --2- 172.21.15.37:0/2759038956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9dc0b2290 0x7ff9dc0b2930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0097a0 tx=0x7ff9cc02a180 comp rx=0 tx=0).stop 2024-04-26T20:22:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.360+0000 7ff9e27c9640 1 -- 172.21.15.37:0/2759038956 shutdown_connections 2024-04-26T20:22:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.360+0000 7ff9e27c9640 1 --2- 172.21.15.37:0/2759038956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0d26f0 0x7ff9dc0cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.360+0000 7ff9e27c9640 1 --2- 172.21.15.37:0/2759038956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9dc0b2290 0x7ff9dc0b2930 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.360+0000 7ff9e27c9640 1 -- 172.21.15.37:0/2759038956 >> 172.21.15.37:0/2759038956 conn(0x7ff9dc0d2ff0 msgr2=0x7ff9dc0d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.360+0000 7ff9e27c9640 1 -- 172.21.15.37:0/2759038956 shutdown_connections 2024-04-26T20:22:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.360+0000 7ff9e27c9640 1 -- 172.21.15.37:0/2759038956 wait complete. 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.361+0000 7ff9e27c9640 1 Processor -- start 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.361+0000 7ff9e27c9640 1 -- start start 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9e27c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc0b2290 0x7ff9dc0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9e27c9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9dc0cf870 0x7ff9dc1595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9e27c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0d26f0 0x7ff9dc0b8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9e27c9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc15ec60 con 0x7ff9dc0cf870 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9e27c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0b2670 con 0x7ff9dc0b2290 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9e27c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc15ede0 con 0x7ff9dc0d26f0 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9dc0cf870 0x7ff9dc1595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9dc0cf870 0x7ff9dc1595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60310/0 (socket says 172.21.15.37:60310) 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9db7fe640 1 -- 172.21.15.37:0/371521403 learned_addr learned my addr 172.21.15.37:0/371521403 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9e0d66640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0d26f0 0x7ff9dc0b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9dbfff640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc0b2290 0x7ff9dc0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9db7fe640 1 -- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0d26f0 msgr2=0x7ff9dc0b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9db7fe640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0d26f0 0x7ff9dc0b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9db7fe640 1 -- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc0b2290 msgr2=0x7ff9dc0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9db7fe640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc0b2290 0x7ff9dc0b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.362+0000 7ff9db7fe640 1 -- 172.21.15.37:0/371521403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc0b5770 con 0x7ff9dc0cf870 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.363+0000 7ff9db7fe640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9dc0cf870 0x7ff9dc1595d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c430 tx=0x7ff9c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.363+0000 7ff9d8ff9640 1 -- 172.21.15.37:0/371521403 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4017070 con 0x7ff9dc0cf870 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.363+0000 7ff9d8ff9640 1 -- 172.21.15.37:0/371521403 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c4005ce0 con 0x7ff9dc0cf870 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.363+0000 7ff9e27c9640 1 -- 172.21.15.37:0/371521403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc1603e0 con 0x7ff9dc0cf870 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.363+0000 7ff9d8ff9640 1 -- 172.21.15.37:0/371521403 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40072f0 con 0x7ff9dc0cf870 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.363+0000 7ff9e27c9640 1 -- 172.21.15.37:0/371521403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc1609a0 con 0x7ff9dc0cf870 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.364+0000 7ff9e27c9640 1 -- 172.21.15.37:0/371521403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9dc06a360 con 0x7ff9dc0cf870 2024-04-26T20:22:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.364+0000 7ff9d8ff9640 1 -- 172.21.15.37:0/371521403 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9c401a070 con 0x7ff9dc0cf870 2024-04-26T20:22:17.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.365+0000 7ff9d8ff9640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff9bc06c8f0 0x7ff9bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.365+0000 7ff9d8ff9640 1 -- 172.21.15.37:0/371521403 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff9c408b030 con 0x7ff9dc0cf870 2024-04-26T20:22:17.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.365+0000 7ff9dbfff640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff9bc06c8f0 0x7ff9bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.365+0000 7ff9dbfff640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff9bc06c8f0 0x7ff9bc06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc034000 tx=0x7ff9cc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:17.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.369+0000 7ff9d8ff9640 1 -- 172.21.15.37:0/371521403 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9dc0b5770 con 0x7ff9dc0cf870 2024-04-26T20:22:17.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.553+0000 7ff9e27c9640 1 -- 172.21.15.37:0/371521403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff9dc0b4380 con 0x7ff9dc0cf870 2024-04-26T20:22:17.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.553+0000 7ff9d8ff9640 1 -- 172.21.15.37:0/371521403 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ff9c4059460 con 0x7ff9dc0cf870 2024-04-26T20:22:17.554 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:17.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 -- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff9bc06c8f0 msgr2=0x7ff9bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff9bc06c8f0 0x7ff9bc06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc034000 tx=0x7ff9cc02e000 comp rx=0 tx=0).stop 2024-04-26T20:22:17.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 -- 172.21.15.37:0/371521403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9dc0cf870 msgr2=0x7ff9dc1595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9dc0cf870 0x7ff9dc1595d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c430 tx=0x7ff9c400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 -- 172.21.15.37:0/371521403 shutdown_connections 2024-04-26T20:22:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff9bc06c8f0 0x7ff9bc06edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0d26f0 0x7ff9dc0b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9dc0cf870 0x7ff9dc1595d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 --2- 172.21.15.37:0/371521403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc0b2290 0x7ff9dc0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 -- 172.21.15.37:0/371521403 >> 172.21.15.37:0/371521403 conn(0x7ff9dc0d2ff0 msgr2=0x7ff9dc0b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 -- 172.21.15.37:0/371521403 shutdown_connections 2024-04-26T20:22:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.555+0000 7ff9e27c9640 1 -- 172.21.15.37:0/371521403 wait complete. 2024-04-26T20:22:17.634 INFO:tasks.cephfs_test_runner:Starting test: test_clone_failure_status_failed (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) ensure failure status is shown when clone is in failed state and validate the reason 2024-04-26T20:22:17.635 INFO:tasks.cephfs_test_runner:test_clone_failure_status_failed (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2024-04-26T20:22:17.635 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:17.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.735+0000 7fbd8e0a9640 1 Processor -- start 2024-04-26T20:22:17.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.735+0000 7fbd8e0a9640 1 -- start start 2024-04-26T20:22:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd8e0a9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbd8815fc10 0x7fbd881600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd8e0a9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd88072e30 con 0x7fbd88159a90 2024-04-26T20:22:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd8e0a9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd88073110 con 0x7fbd880ce9b0 2024-04-26T20:22:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd8e0a9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fbd880733f0 con 0x7fbd8815fc10 2024-04-26T20:22:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd877fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbd88159a90 0x7fbd8806fbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32864/0 (socket says 172.21.15.37:32864) 2024-04-26T20:22:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd877fe640 1 -- 172.21.15.37:0/3676739105 learned_addr learned my addr 172.21.15.37:0/3676739105 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 150996366 0 0) 0x7fbd88072e30 con 0x7fbd88159a90 2024-04-26T20:22:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd70003520 con 0x7fbd88159a90 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd86ffd640 1 --2- 172.21.15.37:0/3676739105 >> v2:172.21.15.134:3300/0 conn(0x7fbd8815fc10 0x7fbd881600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 931564219 0 0) 0x7fbd70003520 con 0x7fbd88159a90 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 >> v2:172.21.15.134:3300/0 conn(0x7fbd8815fc10 msgr2=0x7fbd881600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd867fc640 1 --2- 172.21.15.37:0/3676739105 >> v2:172.21.15.134:3300/0 conn(0x7fbd8815fc10 0x7fbd881600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 >> v1:172.21.15.134:6789/0 conn(0x7fbd880ce9b0 legacy=0x7fbd880cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd880736d0 con 0x7fbd88159a90 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fbd78001670 con 0x7fbd88159a90 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 >> v1:172.21.15.37:6789/0 conn(0x7fbd88159a90 legacy=0x7fbd8806fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.736+0000 7fbd867fc640 1 --2- 172.21.15.37:0/3676739105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd700039b0 0x7fbd70003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd867fc640 1 --2- 172.21.15.37:0/3676739105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd70004800 0x7fbd70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd867fc640 1 --2- 172.21.15.37:0/3676739105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd700052e0 0x7fbd7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd880733f0 con 0x7fbd70004800 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7000c230 con 0x7fbd700039b0 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7000c430 con 0x7fbd700052e0 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd86ffd640 1 --2- 172.21.15.37:0/3676739105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd70004800 0x7fbd70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd877fe640 1 --2- 172.21.15.37:0/3676739105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd700039b0 0x7fbd70003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd87fff640 1 --2- 172.21.15.37:0/3676739105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd700052e0 0x7fbd7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd86ffd640 1 -- 172.21.15.37:0/3676739105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd700052e0 msgr2=0x7fbd7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd86ffd640 1 --2- 172.21.15.37:0/3676739105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd700052e0 0x7fbd7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd86ffd640 1 -- 172.21.15.37:0/3676739105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd700039b0 msgr2=0x7fbd70003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd86ffd640 1 --2- 172.21.15.37:0/3676739105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd700039b0 0x7fbd70003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd86ffd640 1 -- 172.21.15.37:0/3676739105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd70003520 con 0x7fbd70004800 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.737+0000 7fbd87fff640 1 -- 172.21.15.37:0/3676739105 reap_dead start 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.738+0000 7fbd86ffd640 1 --2- 172.21.15.37:0/3676739105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd70004800 0x7fbd70004c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00ba00 tx=0x7fbd70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ceb9bbccccf17ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.738+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c030070 con 0x7fbd70004800 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.738+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7c02ada0 con 0x7fbd70004800 2024-04-26T20:22:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.738+0000 7fbd867fc640 1 -- 172.21.15.37:0/3676739105 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c00b600 con 0x7fbd70004800 2024-04-26T20:22:17.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.738+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/3676739105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd70004800 msgr2=0x7fbd70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.738+0000 7fbd8e0a9640 1 --2- 172.21.15.37:0/3676739105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd70004800 0x7fbd70004c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00ba00 tx=0x7fbd70005830 comp rx=0 tx=0).stop 2024-04-26T20:22:17.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.738+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/3676739105 shutdown_connections 2024-04-26T20:22:17.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.738+0000 7fbd8e0a9640 1 --2- 172.21.15.37:0/3676739105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd70004800 0x7fbd70004c80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.738+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/3676739105 >> 172.21.15.37:0/3676739105 conn(0x7fbd880d0d50 msgr2=0x7fbd880d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:17.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.738+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/3676739105 shutdown_connections 2024-04-26T20:22:17.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.738+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/3676739105 wait complete. 2024-04-26T20:22:17.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.739+0000 7fbd8e0a9640 1 Processor -- start 2024-04-26T20:22:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.740+0000 7fbd8e0a9640 1 -- start start 2024-04-26T20:22:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.740+0000 7fbd8e0a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd70004800 0x7fbd8814ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.740+0000 7fbd8e0a9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd8815a170 0x7fbd8815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.740+0000 7fbd8e0a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd880cc1b0 0x7fbd880cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.740+0000 7fbd8e0a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88159ff0 con 0x7fbd880cc1b0 2024-04-26T20:22:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.740+0000 7fbd8e0a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88159e70 con 0x7fbd70004800 2024-04-26T20:22:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.740+0000 7fbd8e0a9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbd881600c0 con 0x7fbd8815a170 2024-04-26T20:22:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd880cc1b0 0x7fbd880cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd880cc1b0 0x7fbd880cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60326/0 (socket says 172.21.15.37:60326) 2024-04-26T20:22:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd87fff640 1 -- 172.21.15.37:0/81994835 learned_addr learned my addr 172.21.15.37:0/81994835 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:17.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd877fe640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd70004800 0x7fbd8814ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd86ffd640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd8815a170 0x7fbd8815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd87fff640 1 -- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd8815a170 msgr2=0x7fbd8815a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd87fff640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd8815a170 0x7fbd8815a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd87fff640 1 -- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd70004800 msgr2=0x7fbd8814ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd87fff640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd70004800 0x7fbd8814ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd87fff640 1 -- 172.21.15.37:0/81994835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8815abf0 con 0x7fbd880cc1b0 2024-04-26T20:22:17.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd877fe640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd70004800 0x7fbd8814ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:17.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd87fff640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd880cc1b0 0x7fbd880cc630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000f8b0 tx=0x7fbd8000fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:17.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd6ffff640 1 -- 172.21.15.37:0/81994835 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000d7e0 con 0x7fbd880cc1b0 2024-04-26T20:22:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd6ffff640 1 -- 172.21.15.37:0/81994835 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd8000d980 con 0x7fbd880cc1b0 2024-04-26T20:22:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd6ffff640 1 -- 172.21.15.37:0/81994835 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000db50 con 0x7fbd880cc1b0 2024-04-26T20:22:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/81994835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd880b4bc0 con 0x7fbd880cc1b0 2024-04-26T20:22:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.741+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/81994835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd880b5180 con 0x7fbd880cc1b0 2024-04-26T20:22:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.742+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/81994835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd880b5380 con 0x7fbd880cc1b0 2024-04-26T20:22:17.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.743+0000 7fbd6ffff640 1 -- 172.21.15.37:0/81994835 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd8001c020 con 0x7fbd880cc1b0 2024-04-26T20:22:17.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.743+0000 7fbd6ffff640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbd7406c920 0x7fbd7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.743+0000 7fbd6ffff640 1 -- 172.21.15.37:0/81994835 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbd8008b450 con 0x7fbd880cc1b0 2024-04-26T20:22:17.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.743+0000 7fbd877fe640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbd7406c920 0x7fbd7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.744+0000 7fbd877fe640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbd7406c920 0x7fbd7406ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbd78001120 tx=0x7fbd78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:17.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.747+0000 7fbd6ffff640 1 -- 172.21.15.37:0/81994835 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbd880b5380 con 0x7fbd880cc1b0 2024-04-26T20:22:17.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.931+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/81994835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd8806a360 con 0x7fbd880cc1b0 2024-04-26T20:22:17.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.932+0000 7fbd6ffff640 1 -- 172.21.15.37:0/81994835 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7fbd80023020 con 0x7fbd880cc1b0 2024-04-26T20:22:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:17.933 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbd7406c920 msgr2=0x7fbd7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbd7406c920 0x7fbd7406ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbd78001120 tx=0x7fbd78009040 comp rx=0 tx=0).stop 2024-04-26T20:22:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/81994835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd880cc1b0 msgr2=0x7fbd880cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd880cc1b0 0x7fbd880cc630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000f8b0 tx=0x7fbd8000fd70 comp rx=0 tx=0).stop 2024-04-26T20:22:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/81994835 shutdown_connections 2024-04-26T20:22:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd880cc1b0 0x7fbd880cc630 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbd7406c920 0x7fbd7406ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd8815a170 0x7fbd8815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 --2- 172.21.15.37:0/81994835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd70004800 0x7fbd8814ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/81994835 >> 172.21.15.37:0/81994835 conn(0x7fbd880d0d50 msgr2=0x7fbd88158250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/81994835 shutdown_connections 2024-04-26T20:22:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:17.934+0000 7fbd8e0a9640 1 -- 172.21.15.37:0/81994835 wait complete. 2024-04-26T20:22:17.946 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:17.947 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:18.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.048+0000 7fd9e54cc640 1 Processor -- start 2024-04-26T20:22:18.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.048+0000 7fd9e54cc640 1 -- start start 2024-04-26T20:22:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.049+0000 7fd9e54cc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd9e00b0ff0 0x7fd9e006c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.049+0000 7fd9e54cc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e0070920 con 0x7fd9e0070170 2024-04-26T20:22:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.049+0000 7fd9e54cc640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e0070c00 con 0x7fd9e006cdd0 2024-04-26T20:22:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.049+0000 7fd9e54cc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e00cfdf0 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.049+0000 7fd9de7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd9e00b0ff0 0x7fd9e006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.049+0000 7fd9deffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd9e0070170 0x7fd9e006ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32868/0 (socket says 172.21.15.37:32868) 2024-04-26T20:22:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.049+0000 7fd9deffd640 1 -- 172.21.15.37:0/284989661 learned_addr learned my addr 172.21.15.37:0/284989661 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.050+0000 7fd9de7fc640 1 -- 172.21.15.37:0/284989661 >> v1:172.21.15.134:6789/0 conn(0x7fd9e006cdd0 legacy=0x7fd9e0073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.050+0000 7fd9de7fc640 1 -- 172.21.15.37:0/284989661 >> v1:172.21.15.37:6789/0 conn(0x7fd9e0070170 legacy=0x7fd9e006ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.050+0000 7fd9de7fc640 1 -- 172.21.15.37:0/284989661 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e0070550 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.050+0000 7fd9ddffb640 1 -- 172.21.15.37:0/284989661 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2432887272 0 0) 0x7fd9e0070920 con 0x7fd9e0070170 2024-04-26T20:22:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.050+0000 7fd9de7fc640 1 --2- 172.21.15.37:0/284989661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e00b0ff0 0x7fd9e006c890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80097a0 tx=0x7fd9c802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b62f0b08bc548db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.051+0000 7fd9ddffb640 1 -- 172.21.15.37:0/284989661 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8037070 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.051+0000 7fd9ddffb640 1 -- 172.21.15.37:0/284989661 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c802adc0 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.051+0000 7fd9ddffb640 1 -- 172.21.15.37:0/284989661 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8032660 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.051+0000 7fd9e54cc640 1 -- 172.21.15.37:0/284989661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e00b0ff0 msgr2=0x7fd9e006c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.051+0000 7fd9e54cc640 1 --2- 172.21.15.37:0/284989661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e00b0ff0 0x7fd9e006c890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80097a0 tx=0x7fd9c802a0b0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.051+0000 7fd9e54cc640 1 -- 172.21.15.37:0/284989661 shutdown_connections 2024-04-26T20:22:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.051+0000 7fd9e54cc640 1 --2- 172.21.15.37:0/284989661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e00b0ff0 0x7fd9e006c890 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.051+0000 7fd9e54cc640 1 -- 172.21.15.37:0/284989661 >> 172.21.15.37:0/284989661 conn(0x7fd9e00d1e80 msgr2=0x7fd9e00d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.051+0000 7fd9e54cc640 1 -- 172.21.15.37:0/284989661 shutdown_connections 2024-04-26T20:22:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.051+0000 7fd9e54cc640 1 -- 172.21.15.37:0/284989661 wait complete. 2024-04-26T20:22:18.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.053+0000 7fd9e54cc640 1 Processor -- start 2024-04-26T20:22:18.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.053+0000 7fd9e54cc640 1 -- start start 2024-04-26T20:22:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.053+0000 7fd9e54cc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9e006cdd0 0x7fd9e0158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.053+0000 7fd9e54cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9e0070170 0x7fd9e0157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.053+0000 7fd9e54cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e00b0ff0 0x7fd9e015c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.053+0000 7fd9e54cc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e006c400 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.053+0000 7fd9e54cc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0070ee0 con 0x7fd9e0070170 2024-04-26T20:22:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.053+0000 7fd9e54cc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e006f510 con 0x7fd9e006cdd0 2024-04-26T20:22:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.053+0000 7fd9df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e00b0ff0 0x7fd9e015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.053+0000 7fd9df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e00b0ff0 0x7fd9e015c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60332/0 (socket says 172.21.15.37:60332) 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.053+0000 7fd9df7fe640 1 -- 172.21.15.37:0/1542246393 learned_addr learned my addr 172.21.15.37:0/1542246393 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9deffd640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9e006cdd0 0x7fd9e0158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9df7fe640 1 -- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9e006cdd0 msgr2=0x7fd9e0158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9df7fe640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9e006cdd0 0x7fd9e0158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9df7fe640 1 -- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9e0070170 msgr2=0x7fd9e0157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9df7fe640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9e0070170 0x7fd9e0157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9df7fe640 1 -- 172.21.15.37:0/1542246393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e015c990 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9deffd640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9e006cdd0 0x7fd9e0158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9df7fe640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e00b0ff0 0x7fd9e015c140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000b970 tx=0x7fd9d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9d77fe640 1 -- 172.21.15.37:0/1542246393 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0019070 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9d77fe640 1 -- 172.21.15.37:0/1542246393 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d000cb40 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9e54cc640 1 -- 172.21.15.37:0/1542246393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e015cc70 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9d77fe640 1 -- 172.21.15.37:0/1542246393 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0014440 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.054+0000 7fd9e54cc640 1 -- 172.21.15.37:0/1542246393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e015d510 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.055+0000 7fd9e54cc640 1 -- 172.21.15.37:0/1542246393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e015d060 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.056+0000 7fd9d77fe640 1 -- 172.21.15.37:0/1542246393 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9d001c070 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.056+0000 7fd9d77fe640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd9c006c8f0 0x7fd9c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.056+0000 7fd9d77fe640 1 -- 172.21.15.37:0/1542246393 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd9d008b810 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.056+0000 7fd9deffd640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd9c006c8f0 0x7fd9c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.057+0000 7fd9deffd640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd9c006c8f0 0x7fd9c006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8000f60 tx=0x7fd9d8007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:18.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.060+0000 7fd9d77fe640 1 -- 172.21.15.37:0/1542246393 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd9d0059bb0 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.245+0000 7fd9e54cc640 1 -- 172.21.15.37:0/1542246393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd9e00693e0 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.246+0000 7fd9d77fe640 1 -- 172.21.15.37:0/1542246393 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd9d00594b0 con 0x7fd9e00b0ff0 2024-04-26T20:22:18.247 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.247+0000 7fd9e54cc640 1 -- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd9c006c8f0 msgr2=0x7fd9c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.247+0000 7fd9e54cc640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd9c006c8f0 0x7fd9c006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8000f60 tx=0x7fd9d8007ca0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.247+0000 7fd9e54cc640 1 -- 172.21.15.37:0/1542246393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e00b0ff0 msgr2=0x7fd9e015c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.247+0000 7fd9e54cc640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e00b0ff0 0x7fd9e015c140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000b970 tx=0x7fd9d000be30 comp rx=0 tx=0).stop 2024-04-26T20:22:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.248+0000 7fd9e54cc640 1 -- 172.21.15.37:0/1542246393 shutdown_connections 2024-04-26T20:22:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.248+0000 7fd9e54cc640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e00b0ff0 0x7fd9e015c140 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.248+0000 7fd9e54cc640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9e0070170 0x7fd9e0157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.248+0000 7fd9e54cc640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd9c006c8f0 0x7fd9c006edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.248+0000 7fd9e54cc640 1 --2- 172.21.15.37:0/1542246393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9e006cdd0 0x7fd9e0158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.248+0000 7fd9e54cc640 1 -- 172.21.15.37:0/1542246393 >> 172.21.15.37:0/1542246393 conn(0x7fd9e00d1e80 msgr2=0x7fd9e00b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.248+0000 7fd9e54cc640 1 -- 172.21.15.37:0/1542246393 shutdown_connections 2024-04-26T20:22:18.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.248+0000 7fd9e54cc640 1 -- 172.21.15.37:0/1542246393 wait complete. 2024-04-26T20:22:18.259 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:18.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.361+0000 7feb20dbf640 1 Processor -- start 2024-04-26T20:22:18.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.361+0000 7feb20dbf640 1 -- start start 2024-04-26T20:22:18.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.362+0000 7feb20dbf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb1c1610c0 0x7feb1c0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.362+0000 7feb20dbf640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7feb1c0d2150 0x7feb1c0d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.362+0000 7feb20dbf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb1c161510 con 0x7feb1c160800 2024-04-26T20:22:18.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.362+0000 7feb20dbf640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7feb1c0d2590 con 0x7feb1c0d2150 2024-04-26T20:22:18.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.362+0000 7feb20dbf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feb1c0d0930 con 0x7feb1c1610c0 2024-04-26T20:22:18.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.362+0000 7feb1a59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7feb1c160800 0x7feb1c0d5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32878/0 (socket says 172.21.15.37:32878) 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.362+0000 7feb1a59c640 1 -- 172.21.15.37:0/2824021476 learned_addr learned my addr 172.21.15.37:0/2824021476 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1ad9d640 1 --2- 172.21.15.37:0/2824021476 >> v2:172.21.15.134:3300/0 conn(0x7feb1c0d2150 0x7feb1c0d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb19d9b640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb1c1610c0 0x7feb1c0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4118250867 0 0) 0x7feb1c161510 con 0x7feb1c160800 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb08003520 con 0x7feb1c160800 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4098975281 0 0) 0x7feb08003520 con 0x7feb1c160800 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb1c1610c0 msgr2=0x7feb1c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb1c1610c0 0x7feb1c0d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 >> v2:172.21.15.134:3300/0 conn(0x7feb1c0d2150 msgr2=0x7feb1c0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 --2- 172.21.15.37:0/2824021476 >> v2:172.21.15.134:3300/0 conn(0x7feb1c0d2150 0x7feb1c0d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c0d0c00 con 0x7feb1c160800 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7feb14001670 con 0x7feb1c160800 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 >> v1:172.21.15.37:6789/0 conn(0x7feb1c160800 legacy=0x7feb1c0d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb080039b0 0x7feb08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb08004800 0x7feb08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb1c1610c0 0x7feb080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c0d2590 con 0x7feb08004800 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c0d0930 con 0x7feb080039b0 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feb0800b9c0 con 0x7feb1c1610c0 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb19d9b640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb08004800 0x7feb08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1a59c640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb080039b0 0x7feb08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb1ad9d640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb1c1610c0 0x7feb080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb19d9b640 1 -- 172.21.15.37:0/2824021476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb1c1610c0 msgr2=0x7feb080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb19d9b640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb1c1610c0 0x7feb080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb19d9b640 1 -- 172.21.15.37:0/2824021476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb080039b0 msgr2=0x7feb08003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb19d9b640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb080039b0 0x7feb08003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.363+0000 7feb19d9b640 1 -- 172.21.15.37:0/2824021476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb08003520 con 0x7feb08004800 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.364+0000 7feb19d9b640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb08004800 0x7feb08004c80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feb04009ac0 tx=0x7feb08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bb14330f6a97b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:18.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.364+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04036070 con 0x7feb08004800 2024-04-26T20:22:18.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.364+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb04031400 con 0x7feb08004800 2024-04-26T20:22:18.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.364+0000 7feb1959a640 1 -- 172.21.15.37:0/2824021476 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04009690 con 0x7feb08004800 2024-04-26T20:22:18.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.365+0000 7feb20dbf640 1 -- 172.21.15.37:0/2824021476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb08004800 msgr2=0x7feb08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.365+0000 7feb20dbf640 1 --2- 172.21.15.37:0/2824021476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb08004800 0x7feb08004c80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feb04009ac0 tx=0x7feb08005830 comp rx=0 tx=0).stop 2024-04-26T20:22:18.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.365+0000 7feb1ad9d640 1 -- 172.21.15.37:0/2824021476 reap_dead start 2024-04-26T20:22:18.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.365+0000 7feb20dbf640 1 -- 172.21.15.37:0/2824021476 shutdown_connections 2024-04-26T20:22:18.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.365+0000 7feb20dbf640 1 -- 172.21.15.37:0/2824021476 >> 172.21.15.37:0/2824021476 conn(0x7feb1c09e550 msgr2=0x7feb1c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:18.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.365+0000 7feb20dbf640 1 -- 172.21.15.37:0/2824021476 shutdown_connections 2024-04-26T20:22:18.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.365+0000 7feb20dbf640 1 -- 172.21.15.37:0/2824021476 wait complete. 2024-04-26T20:22:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.366+0000 7feb20dbf640 1 Processor -- start 2024-04-26T20:22:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.366+0000 7feb20dbf640 1 -- start start 2024-04-26T20:22:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.367+0000 7feb20dbf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb08004800 0x7feb1c0ba420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.367+0000 7feb20dbf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb1c0b9240 0x7feb1c157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.367+0000 7feb20dbf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb1c157fd0 0x7feb1c158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.367+0000 7feb20dbf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c0d2e10 con 0x7feb1c157fd0 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.367+0000 7feb20dbf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c0d11c0 con 0x7feb1c0b9240 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.367+0000 7feb20dbf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feb1c0d30f0 con 0x7feb08004800 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.367+0000 7feb1ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb1c157fd0 0x7feb1c158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.367+0000 7feb1ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb1c157fd0 0x7feb1c158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60350/0 (socket says 172.21.15.37:60350) 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.367+0000 7feb1ad9d640 1 -- 172.21.15.37:0/362899267 learned_addr learned my addr 172.21.15.37:0/362899267 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.367+0000 7feb1a59c640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb08004800 0x7feb1c0ba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.367+0000 7feb19d9b640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb1c0b9240 0x7feb1c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb1ad9d640 1 -- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb08004800 msgr2=0x7feb1c0ba420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb1ad9d640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb08004800 0x7feb1c0ba420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb1ad9d640 1 -- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb1c0b9240 msgr2=0x7feb1c157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb1ad9d640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb1c0b9240 0x7feb1c157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb1ad9d640 1 -- 172.21.15.37:0/362899267 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c0b9fe0 con 0x7feb1c157fd0 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb1a59c640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb08004800 0x7feb1c0ba420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb19d9b640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb1c0b9240 0x7feb1c157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb1ad9d640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb1c157fd0 0x7feb1c158480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb1c1614a0 tx=0x7feb0c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb12ffd640 1 -- 172.21.15.37:0/362899267 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c00fde0 con 0x7feb1c157fd0 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb12ffd640 1 -- 172.21.15.37:0/362899267 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb0c01d070 con 0x7feb1c157fd0 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb20dbf640 1 -- 172.21.15.37:0/362899267 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c157d40 con 0x7feb1c157fd0 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb12ffd640 1 -- 172.21.15.37:0/362899267 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c015670 con 0x7feb1c157fd0 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.368+0000 7feb20dbf640 1 -- 172.21.15.37:0/362899267 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c0b9a20 con 0x7feb1c157fd0 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.369+0000 7feb20dbf640 1 -- 172.21.15.37:0/362899267 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb1c06a360 con 0x7feb1c157fd0 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.369+0000 7feb12ffd640 1 -- 172.21.15.37:0/362899267 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb0c015810 con 0x7feb1c157fd0 2024-04-26T20:22:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.370+0000 7feb12ffd640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feafc06c920 0x7feafc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.370+0000 7feb12ffd640 1 -- 172.21.15.37:0/362899267 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feb0c08c4b0 con 0x7feb1c157fd0 2024-04-26T20:22:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.370+0000 7feb1a59c640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feafc06c920 0x7feafc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.371+0000 7feb1a59c640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feafc06c920 0x7feafc06ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feb14000fb0 tx=0x7feb14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.374+0000 7feb12ffd640 1 -- 172.21.15.37:0/362899267 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb0c010040 con 0x7feb1c157fd0 2024-04-26T20:22:18.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.559+0000 7feb20dbf640 1 -- 172.21.15.37:0/362899267 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feb1c0d0100 con 0x7feb1c157fd0 2024-04-26T20:22:18.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.560+0000 7feb12ffd640 1 -- 172.21.15.37:0/362899267 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7feb0c05a200 con 0x7feb1c157fd0 2024-04-26T20:22:18.561 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 -- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feafc06c920 msgr2=0x7feafc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feafc06c920 0x7feafc06ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feb14000fb0 tx=0x7feb14009040 comp rx=0 tx=0).stop 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 -- 172.21.15.37:0/362899267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb1c157fd0 msgr2=0x7feb1c158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb1c157fd0 0x7feb1c158480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb1c1614a0 tx=0x7feb0c00f580 comp rx=0 tx=0).stop 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 -- 172.21.15.37:0/362899267 shutdown_connections 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb1c157fd0 0x7feb1c158480 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb1c0b9240 0x7feb1c157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7feafc06c920 0x7feafc06ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 --2- 172.21.15.37:0/362899267 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb08004800 0x7feb1c0ba420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 -- 172.21.15.37:0/362899267 >> 172.21.15.37:0/362899267 conn(0x7feb1c09e550 msgr2=0x7feb1c0b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 -- 172.21.15.37:0/362899267 shutdown_connections 2024-04-26T20:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.561+0000 7feb20dbf640 1 -- 172.21.15.37:0/362899267 wait complete. 2024-04-26T20:22:18.573 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:18.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.675+0000 7ffaa70d9640 1 Processor -- start 2024-04-26T20:22:18.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.675+0000 7ffaa70d9640 1 -- start start 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.676+0000 7ffaa70d9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffaa015f470 0x7ffaa00d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.676+0000 7ffaa70d9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffaa00d21b0 0x7ffaa00b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.676+0000 7ffaa70d9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa00d2ed0 con 0x7ffaa00d0d30 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.676+0000 7ffaa70d9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa00d1190 con 0x7ffaa00d21b0 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.676+0000 7ffaa70d9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa00d31b0 con 0x7ffaa015f470 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.676+0000 7ffaa5676640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffaa00d21b0 0x7ffaa00b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.676+0000 7ffa9ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ffaa00d0d30 0x7ffaa00d1aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32884/0 (socket says 172.21.15.37:32884) 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.676+0000 7ffa9ffff640 1 -- 172.21.15.37:0/2688522227 learned_addr learned my addr 172.21.15.37:0/2688522227 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.676+0000 7ffaa4e75640 1 --2- 172.21.15.37:0/2688522227 >> v2:172.21.15.134:3300/0 conn(0x7ffaa015f470 0x7ffaa00d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.677+0000 7ffaa5676640 1 -- 172.21.15.37:0/2688522227 >> v2:172.21.15.134:3300/0 conn(0x7ffaa015f470 msgr2=0x7ffaa00d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.677+0000 7ffaa5676640 1 --2- 172.21.15.37:0/2688522227 >> v2:172.21.15.134:3300/0 conn(0x7ffaa015f470 0x7ffaa00d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.677+0000 7ffaa5676640 1 -- 172.21.15.37:0/2688522227 >> v1:172.21.15.37:6789/0 conn(0x7ffaa00d0d30 legacy=0x7ffaa00d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.677+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/2688522227 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013486234 0 0) 0x7ffaa00d2ed0 con 0x7ffaa00d0d30 2024-04-26T20:22:18.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.677+0000 7ffaa5676640 1 -- 172.21.15.37:0/2688522227 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa00d3490 con 0x7ffaa00d21b0 2024-04-26T20:22:18.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.677+0000 7ffaa4e75640 1 --2- 172.21.15.37:0/2688522227 >> v2:172.21.15.134:3300/0 conn(0x7ffaa015f470 0x7ffaa00d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:18.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.677+0000 7ffaa5676640 1 --2- 172.21.15.37:0/2688522227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa00d21b0 0x7ffaa00b7fb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00d61a0 tx=0x7ffa8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584c44328cf2ef44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:18.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.677+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/2688522227 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88037070 con 0x7ffaa00d21b0 2024-04-26T20:22:18.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.677+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/2688522227 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa88032440 con 0x7ffaa00d21b0 2024-04-26T20:22:18.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.677+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/2688522227 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88032810 con 0x7ffaa00d21b0 2024-04-26T20:22:18.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.678+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2688522227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa00d21b0 msgr2=0x7ffaa00b7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.678+0000 7ffaa70d9640 1 --2- 172.21.15.37:0/2688522227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa00d21b0 0x7ffaa00b7fb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00d61a0 tx=0x7ffa8802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.678+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2688522227 shutdown_connections 2024-04-26T20:22:18.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.678+0000 7ffaa70d9640 1 --2- 172.21.15.37:0/2688522227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa00d21b0 0x7ffaa00b7fb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.678+0000 7ffaa70d9640 1 --2- 172.21.15.37:0/2688522227 >> v2:172.21.15.134:3300/0 conn(0x7ffaa015f470 0x7ffaa00d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.678+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2688522227 >> 172.21.15.37:0/2688522227 conn(0x7ffaa009e510 msgr2=0x7ffaa00d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:18.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.678+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2688522227 shutdown_connections 2024-04-26T20:22:18.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.678+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2688522227 wait complete. 2024-04-26T20:22:18.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.679+0000 7ffaa70d9640 1 Processor -- start 2024-04-26T20:22:18.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.679+0000 7ffaa70d9640 1 -- start start 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.680+0000 7ffaa70d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa00d0d30 0x7ffaa00ba900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.680+0000 7ffaa70d9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaa00d21b0 0x7ffaa0155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.680+0000 7ffaa70d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa015f470 0x7ffaa0157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.680+0000 7ffaa70d9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa015f850 con 0x7ffaa00d0d30 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.680+0000 7ffaa70d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa00d3a50 con 0x7ffaa015f470 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.680+0000 7ffaa70d9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa00d3f70 con 0x7ffaa00d21b0 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.680+0000 7ffaa4e75640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa00d0d30 0x7ffaa00ba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.680+0000 7ffaa4e75640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa00d0d30 0x7ffaa00ba900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60358/0 (socket says 172.21.15.37:60358) 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.680+0000 7ffaa4e75640 1 -- 172.21.15.37:0/2778902463 learned_addr learned my addr 172.21.15.37:0/2778902463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.680+0000 7ffaa5676640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa015f470 0x7ffaa0157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.680+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaa00d21b0 0x7ffaa0155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffaa4e75640 1 -- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaa00d21b0 msgr2=0x7ffaa0155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffaa4e75640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaa00d21b0 0x7ffaa0155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffaa4e75640 1 -- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa015f470 msgr2=0x7ffaa0157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffaa4e75640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa015f470 0x7ffaa0157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffaa4e75640 1 -- 172.21.15.37:0/2778902463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa00b7cf0 con 0x7ffaa00d0d30 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffaa5676640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa015f470 0x7ffaa0157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaa00d21b0 0x7ffaa0155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffaa4e75640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa00d0d30 0x7ffaa00ba900 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800d7d0 tx=0x7ffa9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffa9d7fa640 1 -- 172.21.15.37:0/2778902463 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98004030 con 0x7ffaa00d0d30 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffa9d7fa640 1 -- 172.21.15.37:0/2778902463 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa980041d0 con 0x7ffaa00d0d30 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2778902463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa0156750 con 0x7ffaa00d0d30 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffa9d7fa640 1 -- 172.21.15.37:0/2778902463 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9800b6c0 con 0x7ffaa00d0d30 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.681+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2778902463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa00b8290 con 0x7ffaa00d0d30 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.682+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2778902463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa006a360 con 0x7ffaa00d0d30 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.682+0000 7ffa9d7fa640 1 -- 172.21.15.37:0/2778902463 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa9800b860 con 0x7ffaa00d0d30 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.683+0000 7ffa9d7fa640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa8006c8f0 0x7ffa8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.683+0000 7ffa9d7fa640 1 -- 172.21.15.37:0/2778902463 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffa9808bd40 con 0x7ffaa00d0d30 2024-04-26T20:22:18.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.683+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa8006c8f0 0x7ffa8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.684+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa8006c8f0 0x7ffa8006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa90007fd0 tx=0x7ffa90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:18.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.687+0000 7ffa9d7fa640 1 -- 172.21.15.37:0/2778902463 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa9805a200 con 0x7ffaa00d0d30 2024-04-26T20:22:18.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.873+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2778902463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffaa00d5870 con 0x7ffaa00d0d30 2024-04-26T20:22:18.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.873+0000 7ffa9d7fa640 1 -- 172.21.15.37:0/2778902463 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ffa98059b00 con 0x7ffaa00d0d30 2024-04-26T20:22:18.875 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.875+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa8006c8f0 msgr2=0x7ffa8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.875+0000 7ffaa70d9640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa8006c8f0 0x7ffa8006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa90007fd0 tx=0x7ffa90009040 comp rx=0 tx=0).stop 2024-04-26T20:22:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.875+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2778902463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa00d0d30 msgr2=0x7ffaa00ba900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.875+0000 7ffaa70d9640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa00d0d30 0x7ffaa00ba900 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800d7d0 tx=0x7ffa9800dc90 comp rx=0 tx=0).stop 2024-04-26T20:22:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.875+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2778902463 shutdown_connections 2024-04-26T20:22:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.875+0000 7ffaa70d9640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa015f470 0x7ffaa0157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.875+0000 7ffaa70d9640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa8006c8f0 0x7ffa8006edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.875+0000 7ffaa70d9640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaa00d21b0 0x7ffaa0155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.875+0000 7ffaa70d9640 1 --2- 172.21.15.37:0/2778902463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa00d0d30 0x7ffaa00ba900 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.875+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2778902463 >> 172.21.15.37:0/2778902463 conn(0x7ffaa009e510 msgr2=0x7ffaa0160aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.876+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2778902463 shutdown_connections 2024-04-26T20:22:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.876+0000 7ffaa70d9640 1 -- 172.21.15.37:0/2778902463 wait complete. 2024-04-26T20:22:18.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed' 2024-04-26T20:22:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.988+0000 7ff8d2522640 1 Processor -- start 2024-04-26T20:22:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.988+0000 7ff8d2522640 1 -- start start 2024-04-26T20:22:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8d2522640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8cc162e40 0x7ff8cc0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8d2522640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff8cc0b2050 0x7ff8cc0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8d2522640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff8cc0cfd70 0x7ff8cc0ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8d2522640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff8cc163220 con 0x7ff8cc0b2050 2024-04-26T20:22:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8d2522640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff8cc0d0eb0 con 0x7ff8cc0cfd70 2024-04-26T20:22:18.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8d2522640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff8cc0d01a0 con 0x7ff8cc162e40 2024-04-26T20:22:18.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8cb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff8cc0b2050 0x7ff8cc0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8cb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff8cc0b2050 0x7ff8cc0cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60364/0 (socket says 172.21.15.37:60364) 2024-04-26T20:22:18.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8cb7fe640 1 -- 172.21.15.37:0/53594387 learned_addr learned my addr 172.21.15.37:0/53594387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:18.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8cbfff640 1 --2- 172.21.15.37:0/53594387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8cc162e40 0x7ff8cc0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8d0abf640 1 --2- 172.21.15.37:0/53594387 >> v2:172.21.15.134:3300/0 conn(0x7ff8cc0cfd70 0x7ff8cc0ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8cb7fe640 1 -- 172.21.15.37:0/53594387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8cc162e40 msgr2=0x7ff8cc0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:18.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8cb7fe640 1 --2- 172.21.15.37:0/53594387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8cc162e40 0x7ff8cc0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8cb7fe640 1 -- 172.21.15.37:0/53594387 >> v2:172.21.15.134:3300/0 conn(0x7ff8cc0cfd70 msgr2=0x7ff8cc0ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:18.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8cb7fe640 1 --2- 172.21.15.37:0/53594387 >> v2:172.21.15.134:3300/0 conn(0x7ff8cc0cfd70 0x7ff8cc0ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.989+0000 7ff8cb7fe640 1 -- 172.21.15.37:0/53594387 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc0b1d70 con 0x7ff8cc0b2050 2024-04-26T20:22:18.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.990+0000 7ff8cbfff640 1 --2- 172.21.15.37:0/53594387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8cc162e40 0x7ff8cc0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:18.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.990+0000 7ff8cb7fe640 1 --2- 172.21.15.37:0/53594387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8cc0b2050 0x7ff8cc0cf830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc072bb0 tx=0x7ff8bc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e505fcbc4422a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:18.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.990+0000 7ff8caffd640 1 -- 172.21.15.37:0/53594387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc037070 con 0x7ff8cc0b2050 2024-04-26T20:22:18.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.990+0000 7ff8caffd640 1 -- 172.21.15.37:0/53594387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8bc032440 con 0x7ff8cc0b2050 2024-04-26T20:22:18.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.990+0000 7ff8caffd640 1 -- 172.21.15.37:0/53594387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc032810 con 0x7ff8cc0b2050 2024-04-26T20:22:19.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.991+0000 7ff8d2522640 1 -- 172.21.15.37:0/53594387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8cc0b2050 msgr2=0x7ff8cc0cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:19.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.991+0000 7ff8d2522640 1 --2- 172.21.15.37:0/53594387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8cc0b2050 0x7ff8cc0cf830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc072bb0 tx=0x7ff8bc02a140 comp rx=0 tx=0).stop 2024-04-26T20:22:19.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.991+0000 7ff8d2522640 1 -- 172.21.15.37:0/53594387 shutdown_connections 2024-04-26T20:22:19.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.991+0000 7ff8d2522640 1 --2- 172.21.15.37:0/53594387 >> v2:172.21.15.134:3300/0 conn(0x7ff8cc0cfd70 0x7ff8cc0ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:19.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.991+0000 7ff8d2522640 1 --2- 172.21.15.37:0/53594387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8cc0b2050 0x7ff8cc0cf830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:19.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.991+0000 7ff8d2522640 1 --2- 172.21.15.37:0/53594387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8cc162e40 0x7ff8cc0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:19.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.991+0000 7ff8d2522640 1 -- 172.21.15.37:0/53594387 >> 172.21.15.37:0/53594387 conn(0x7ff8cc09ba50 msgr2=0x7ff8cc09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:19.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.991+0000 7ff8d2522640 1 -- 172.21.15.37:0/53594387 shutdown_connections 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.991+0000 7ff8d2522640 1 -- 172.21.15.37:0/53594387 wait complete. 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.992+0000 7ff8d2522640 1 Processor -- start 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.992+0000 7ff8d2522640 1 -- start start 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8d2522640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8cc0b2050 0x7ff8cc0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8d2522640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8cc0cfd70 0x7ff8cc0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8d2522640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8cc162e40 0x7ff8cc1f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8d2522640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc0d1f20 con 0x7ff8cc0b2050 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8d2522640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc0d1c40 con 0x7ff8cc0cfd70 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8d2522640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff8cc0b24b0 con 0x7ff8cc162e40 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8cc0b2050 0x7ff8cc0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8cc0b2050 0x7ff8cc0cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60380/0 (socket says 172.21.15.37:60380) 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8cbfff640 1 -- 172.21.15.37:0/4230588838 learned_addr learned my addr 172.21.15.37:0/4230588838 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:19.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8d0abf640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8cc162e40 0x7ff8cc1f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8cb7fe640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8cc0cfd70 0x7ff8cc0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8cbfff640 1 -- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8cc162e40 msgr2=0x7ff8cc1f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8cbfff640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8cc162e40 0x7ff8cc1f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8cbfff640 1 -- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8cc0cfd70 msgr2=0x7ff8cc0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.993+0000 7ff8cbfff640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8cc0cfd70 0x7ff8cc0ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.994+0000 7ff8cbfff640 1 -- 172.21.15.37:0/4230588838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc0cae00 con 0x7ff8cc0b2050 2024-04-26T20:22:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.994+0000 7ff8cbfff640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8cc0b2050 0x7ff8cc0cc6a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000c820 tx=0x7ff8c000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.994+0000 7ff8c8ff9640 1 -- 172.21.15.37:0/4230588838 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0015070 con 0x7ff8cc0b2050 2024-04-26T20:22:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.994+0000 7ff8c8ff9640 1 -- 172.21.15.37:0/4230588838 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c0007a30 con 0x7ff8cc0b2050 2024-04-26T20:22:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.994+0000 7ff8d2522640 1 -- 172.21.15.37:0/4230588838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc0cb0e0 con 0x7ff8cc0b2050 2024-04-26T20:22:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.994+0000 7ff8c8ff9640 1 -- 172.21.15.37:0/4230588838 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0007e00 con 0x7ff8cc0b2050 2024-04-26T20:22:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.994+0000 7ff8d2522640 1 -- 172.21.15.37:0/4230588838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc0cb6a0 con 0x7ff8cc0b2050 2024-04-26T20:22:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.995+0000 7ff8d2522640 1 -- 172.21.15.37:0/4230588838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc06a360 con 0x7ff8cc0b2050 2024-04-26T20:22:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.995+0000 7ff8c8ff9640 1 -- 172.21.15.37:0/4230588838 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8c0018070 con 0x7ff8cc0b2050 2024-04-26T20:22:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.996+0000 7ff8c8ff9640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff8b806c8f0 0x7ff8b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.996+0000 7ff8c8ff9640 1 -- 172.21.15.37:0/4230588838 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff8c008ba90 con 0x7ff8cc0b2050 2024-04-26T20:22:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.996+0000 7ff8cb7fe640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff8b806c8f0 0x7ff8b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:18.997+0000 7ff8cb7fe640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff8b806c8f0 0x7ff8b806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc034000 tx=0x7ff8bc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:19.000+0000 7ff8c8ff9640 1 -- 172.21.15.37:0/4230588838 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8c0059f50 con 0x7ff8cc0b2050 2024-04-26T20:22:19.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:19.255+0000 7ff8d2522640 1 -- 172.21.15.37:0/4230588838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed"]} v 0) -- 0x7ff8cc15db40 con 0x7ff8cc0b2050 2024-04-26T20:22:19.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:22:19.377 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:22:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.210+0000 7ff8c8ff9640 1 -- 172.21.15.37:0/4230588838 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed"]}]=0 v54) ==== 168+0+0 (secure 0 0 0) 0x7ff8c0059850 con 0x7ff8cc0b2050 2024-04-26T20:22:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 -- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff8b806c8f0 msgr2=0x7ff8b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff8b806c8f0 0x7ff8b806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc034000 tx=0x7ff8bc02e000 comp rx=0 tx=0).stop 2024-04-26T20:22:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 -- 172.21.15.37:0/4230588838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8cc0b2050 msgr2=0x7ff8cc0cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8cc0b2050 0x7ff8cc0cc6a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000c820 tx=0x7ff8c000cc20 comp rx=0 tx=0).stop 2024-04-26T20:22:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 -- 172.21.15.37:0/4230588838 shutdown_connections 2024-04-26T20:22:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff8b806c8f0 0x7ff8b806edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8cc162e40 0x7ff8cc1f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8cc0cfd70 0x7ff8cc0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 --2- 172.21.15.37:0/4230588838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8cc0b2050 0x7ff8cc0cc6a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 -- 172.21.15.37:0/4230588838 >> 172.21.15.37:0/4230588838 conn(0x7ff8cc09ba50 msgr2=0x7ff8cc0cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 -- 172.21.15.37:0/4230588838 shutdown_connections 2024-04-26T20:22:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.213+0000 7ff8d2522640 1 -- 172.21.15.37:0/4230588838 wait complete. 2024-04-26T20:22:20.229 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:22:20.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.342+0000 7fb19e299640 1 Processor -- start 2024-04-26T20:22:20.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.342+0000 7fb19e299640 1 -- start start 2024-04-26T20:22:20.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb19e299640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb19806e700 0x7fb1980ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb19e299640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1980cd100 0x7fb1980ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb19e299640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb19806ece0 con 0x7fb19815d460 2024-04-26T20:22:20.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb19e299640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb1980add80 con 0x7fb19806e700 2024-04-26T20:22:20.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb19e299640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb19806efc0 con 0x7fb1980cd100 2024-04-26T20:22:20.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb1977fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb19806e700 0x7fb1980ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb1977fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb19806e700 0x7fb1980ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60384/0 (socket says 172.21.15.37:60384) 2024-04-26T20:22:20.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb1977fe640 1 -- 172.21.15.37:0/675145715 learned_addr learned my addr 172.21.15.37:0/675145715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:20.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb19c836640 1 --2- 172.21.15.37:0/675145715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1980cd100 0x7fb1980ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb1977fe640 1 -- 172.21.15.37:0/675145715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1980cd100 msgr2=0x7fb1980ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb1977fe640 1 --2- 172.21.15.37:0/675145715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1980cd100 0x7fb1980ae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb1977fe640 1 -- 172.21.15.37:0/675145715 >> v1:172.21.15.134:6789/0 conn(0x7fb19815d460 legacy=0x7fb198073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.343+0000 7fb1977fe640 1 -- 172.21.15.37:0/675145715 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19806f2a0 con 0x7fb19806e700 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.344+0000 7fb1977fe640 1 --2- 172.21.15.37:0/675145715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19806e700 0x7fb1980ccbc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb188009870 tx=0x7fb18802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490a3e3d5ad1098 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.344+0000 7fb196ffd640 1 -- 172.21.15.37:0/675145715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188037070 con 0x7fb19806e700 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.344+0000 7fb196ffd640 1 -- 172.21.15.37:0/675145715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb188032440 con 0x7fb19806e700 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.344+0000 7fb196ffd640 1 -- 172.21.15.37:0/675145715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188032810 con 0x7fb19806e700 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.345+0000 7fb19e299640 1 -- 172.21.15.37:0/675145715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19806e700 msgr2=0x7fb1980ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.345+0000 7fb19e299640 1 --2- 172.21.15.37:0/675145715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19806e700 0x7fb1980ccbc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb188009870 tx=0x7fb18802a2a0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.345+0000 7fb19e299640 1 -- 172.21.15.37:0/675145715 shutdown_connections 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.345+0000 7fb19e299640 1 --2- 172.21.15.37:0/675145715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1980cd100 0x7fb1980ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.345+0000 7fb19e299640 1 --2- 172.21.15.37:0/675145715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19806e700 0x7fb1980ccbc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.345+0000 7fb19e299640 1 -- 172.21.15.37:0/675145715 >> 172.21.15.37:0/675145715 conn(0x7fb1980d0d70 msgr2=0x7fb1980d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:20.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.345+0000 7fb19e299640 1 -- 172.21.15.37:0/675145715 shutdown_connections 2024-04-26T20:22:20.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.345+0000 7fb19e299640 1 -- 172.21.15.37:0/675145715 wait complete. 2024-04-26T20:22:20.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.346+0000 7fb19e299640 1 Processor -- start 2024-04-26T20:22:20.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.346+0000 7fb19e299640 1 -- start start 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb19e299640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19806e700 0x7fb1981f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb19e299640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1980cd100 0x7fb1980c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb19e299640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb19815d460 0x7fb1980b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb19e299640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb19815d9c0 con 0x7fb19806e700 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb19e299640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb19815d840 con 0x7fb19815d460 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb19e299640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1980ae750 con 0x7fb1980cd100 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb197fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19806e700 0x7fb1981f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb197fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19806e700 0x7fb1981f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60386/0 (socket says 172.21.15.37:60386) 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb197fff640 1 -- 172.21.15.37:0/4246732982 learned_addr learned my addr 172.21.15.37:0/4246732982 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb197fff640 1 -- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1980cd100 msgr2=0x7fb1980c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb197fff640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1980cd100 0x7fb1980c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb197fff640 1 -- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb19815d460 msgr2=0x7fb1980b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:20.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb197fff640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb19815d460 0x7fb1980b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb197fff640 1 -- 172.21.15.37:0/4246732982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1980af9a0 con 0x7fb19806e700 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.347+0000 7fb197fff640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19806e700 0x7fb1981f5f50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b3f0 tx=0x7fb18c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.348+0000 7fb194ff9640 1 -- 172.21.15.37:0/4246732982 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c015020 con 0x7fb19806e700 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.348+0000 7fb194ff9640 1 -- 172.21.15.37:0/4246732982 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb18c011830 con 0x7fb19806e700 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.348+0000 7fb19e299640 1 -- 172.21.15.37:0/4246732982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1980afc80 con 0x7fb19806e700 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.348+0000 7fb194ff9640 1 -- 172.21.15.37:0/4246732982 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c011c20 con 0x7fb19806e700 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.348+0000 7fb19e299640 1 -- 172.21.15.37:0/4246732982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1980b0240 con 0x7fb19806e700 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.349+0000 7fb19e299640 1 -- 172.21.15.37:0/4246732982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb19806a360 con 0x7fb19806e700 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.349+0000 7fb194ff9640 1 -- 172.21.15.37:0/4246732982 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb18c018020 con 0x7fb19806e700 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.349+0000 7fb194ff9640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb18406c920 0x7fb18406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.350+0000 7fb194ff9640 1 -- 172.21.15.37:0/4246732982 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb18c08b720 con 0x7fb19806e700 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.350+0000 7fb1977fe640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb18406c920 0x7fb18406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.350+0000 7fb1977fe640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb18406c920 0x7fb18406ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb188034000 tx=0x7fb18802c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:20.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.354+0000 7fb194ff9640 1 -- 172.21.15.37:0/4246732982 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb18c01f030 con 0x7fb19806e700 2024-04-26T20:22:20.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.538+0000 7fb19e299640 1 -- 172.21.15.37:0/4246732982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb19815e9c0 con 0x7fb19806e700 2024-04-26T20:22:20.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.541+0000 7fb194ff9640 1 -- 172.21.15.37:0/4246732982 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fb18c0172c0 con 0x7fb19806e700 2024-04-26T20:22:20.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.543+0000 7fb19e299640 1 -- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb18406c920 msgr2=0x7fb18406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:20.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.543+0000 7fb19e299640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb18406c920 0x7fb18406ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb188034000 tx=0x7fb18802c000 comp rx=0 tx=0).stop 2024-04-26T20:22:20.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.543+0000 7fb19e299640 1 -- 172.21.15.37:0/4246732982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19806e700 msgr2=0x7fb1981f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:20.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.543+0000 7fb19e299640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19806e700 0x7fb1981f5f50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b3f0 tx=0x7fb18c00b8b0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.543+0000 7fb19e299640 1 -- 172.21.15.37:0/4246732982 shutdown_connections 2024-04-26T20:22:20.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.543+0000 7fb19e299640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb19815d460 0x7fb1980b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.543+0000 7fb19e299640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb18406c920 0x7fb18406ede0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.543+0000 7fb19e299640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1980cd100 0x7fb1980c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.543+0000 7fb19e299640 1 --2- 172.21.15.37:0/4246732982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19806e700 0x7fb1981f5f50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.543+0000 7fb19e299640 1 -- 172.21.15.37:0/4246732982 >> 172.21.15.37:0/4246732982 conn(0x7fb1980d0d70 msgr2=0x7fb198073b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:20.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.544+0000 7fb19e299640 1 -- 172.21.15.37:0/4246732982 shutdown_connections 2024-04-26T20:22:20.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.544+0000 7fb19e299640 1 -- 172.21.15.37:0/4246732982 wait complete. 2024-04-26T20:22:20.556 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:22:20.556 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:22:20.612 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:22:20.675 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:22:20.676 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:20.676 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:20.691 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:20.691 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:20.747 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:20.748 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:20.804 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:20.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.944+0000 7faa528cc640 1 Processor -- start 2024-04-26T20:22:20.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.944+0000 7faa528cc640 1 -- start start 2024-04-26T20:22:20.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.945+0000 7faa528cc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faa4c0d3820 0x7faa4c0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.945+0000 7faa528cc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faa4c0b4ba0 0x7faa4c0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.945+0000 7faa528cc640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa4c0d0210 con 0x7faa4c162b40 2024-04-26T20:22:20.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.945+0000 7faa528cc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7faa4c0d41c0 con 0x7faa4c0b4ba0 2024-04-26T20:22:20.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.945+0000 7faa528cc640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7faa4c0d04f0 con 0x7faa4c0d3820 2024-04-26T20:22:20.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.945+0000 7faa50e69640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faa4c0b4ba0 0x7faa4c0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.945+0000 7faa50e69640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faa4c0b4ba0 0x7faa4c0b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60400/0 (socket says 172.21.15.37:60400) 2024-04-26T20:22:20.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.945+0000 7faa50e69640 1 -- 172.21.15.37:0/3839841185 learned_addr learned my addr 172.21.15.37:0/3839841185 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:20.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.946+0000 7faa50e69640 1 -- 172.21.15.37:0/3839841185 >> v2:172.21.15.134:3300/0 conn(0x7faa4c0d3820 msgr2=0x7faa4c0d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:20.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.946+0000 7faa4b7fe640 1 --2- 172.21.15.37:0/3839841185 >> v2:172.21.15.134:3300/0 conn(0x7faa4c0d3820 0x7faa4c0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.946+0000 7faa50e69640 1 --2- 172.21.15.37:0/3839841185 >> v2:172.21.15.134:3300/0 conn(0x7faa4c0d3820 0x7faa4c0d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.946+0000 7faa50e69640 1 -- 172.21.15.37:0/3839841185 >> v1:172.21.15.134:6789/0 conn(0x7faa4c162b40 legacy=0x7faa4c0d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:20.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.946+0000 7faa50e69640 1 -- 172.21.15.37:0/3839841185 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c0d07d0 con 0x7faa4c0b4ba0 2024-04-26T20:22:20.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.946+0000 7faa50e69640 1 --2- 172.21.15.37:0/3839841185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa4c0b4ba0 0x7faa4c0b5030 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faa4c08f8e0 tx=0x7faa4402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b998b08ac8839fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:20.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.947+0000 7faa4affd640 1 -- 172.21.15.37:0/3839841185 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44036070 con 0x7faa4c0b4ba0 2024-04-26T20:22:20.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.947+0000 7faa4affd640 1 -- 172.21.15.37:0/3839841185 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa44031600 con 0x7faa4c0b4ba0 2024-04-26T20:22:20.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.947+0000 7faa4affd640 1 -- 172.21.15.37:0/3839841185 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440319d0 con 0x7faa4c0b4ba0 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.947+0000 7faa528cc640 1 -- 172.21.15.37:0/3839841185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa4c0b4ba0 msgr2=0x7faa4c0b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.947+0000 7faa528cc640 1 --2- 172.21.15.37:0/3839841185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa4c0b4ba0 0x7faa4c0b5030 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faa4c08f8e0 tx=0x7faa4402a6f0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.947+0000 7faa528cc640 1 -- 172.21.15.37:0/3839841185 shutdown_connections 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.947+0000 7faa528cc640 1 --2- 172.21.15.37:0/3839841185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa4c0b4ba0 0x7faa4c0b5030 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.947+0000 7faa528cc640 1 --2- 172.21.15.37:0/3839841185 >> v2:172.21.15.134:3300/0 conn(0x7faa4c0d3820 0x7faa4c0d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.947+0000 7faa528cc640 1 -- 172.21.15.37:0/3839841185 >> 172.21.15.37:0/3839841185 conn(0x7faa4c09e550 msgr2=0x7faa4c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.947+0000 7faa528cc640 1 -- 172.21.15.37:0/3839841185 shutdown_connections 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.947+0000 7faa528cc640 1 -- 172.21.15.37:0/3839841185 wait complete. 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.948+0000 7faa528cc640 1 Processor -- start 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa528cc640 1 -- start start 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa528cc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa4c0b4ba0 0x7faa4c0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa528cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa4c0d3820 0x7faa4c0ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa528cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa4c162b40 0x7faa4c14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa528cc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c0d12b0 con 0x7faa4c0d3820 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa528cc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c0d0d90 con 0x7faa4c162b40 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa528cc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faa4c0cfe30 con 0x7faa4c0b4ba0 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa4b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa4c0d3820 0x7faa4c0ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa4b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa4c0d3820 0x7faa4c0ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60410/0 (socket says 172.21.15.37:60410) 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa4b7fe640 1 -- 172.21.15.37:0/3607768313 learned_addr learned my addr 172.21.15.37:0/3607768313 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa50e69640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa4c162b40 0x7faa4c14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.949+0000 7faa4bfff640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa4c0b4ba0 0x7faa4c0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.950+0000 7faa4b7fe640 1 -- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa4c0b4ba0 msgr2=0x7faa4c0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.950+0000 7faa4b7fe640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa4c0b4ba0 0x7faa4c0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.950+0000 7faa4b7fe640 1 -- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa4c162b40 msgr2=0x7faa4c14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.950+0000 7faa4b7fe640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa4c162b40 0x7faa4c14bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.950+0000 7faa4b7fe640 1 -- 172.21.15.37:0/3607768313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c0cb0a0 con 0x7faa4c0d3820 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.950+0000 7faa4b7fe640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa4c0d3820 0x7faa4c0ca850 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00c820 tx=0x7faa3c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.950+0000 7faa48ff9640 1 -- 172.21.15.37:0/3607768313 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c015070 con 0x7faa4c0d3820 2024-04-26T20:22:20.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.950+0000 7faa48ff9640 1 -- 172.21.15.37:0/3607768313 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa3c004020 con 0x7faa4c0d3820 2024-04-26T20:22:20.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.950+0000 7faa528cc640 1 -- 172.21.15.37:0/3607768313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4c0cb380 con 0x7faa4c0d3820 2024-04-26T20:22:20.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.950+0000 7faa48ff9640 1 -- 172.21.15.37:0/3607768313 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0043f0 con 0x7faa4c0d3820 2024-04-26T20:22:20.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.950+0000 7faa528cc640 1 -- 172.21.15.37:0/3607768313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4c0cb940 con 0x7faa4c0d3820 2024-04-26T20:22:20.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.951+0000 7faa528cc640 1 -- 172.21.15.37:0/3607768313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4c06a360 con 0x7faa4c0d3820 2024-04-26T20:22:20.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.952+0000 7faa48ff9640 1 -- 172.21.15.37:0/3607768313 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa3c007830 con 0x7faa4c0d3820 2024-04-26T20:22:20.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.952+0000 7faa48ff9640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faa3806c8f0 0x7faa3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.952+0000 7faa48ff9640 1 -- 172.21.15.37:0/3607768313 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faa3c08b8b0 con 0x7faa4c0d3820 2024-04-26T20:22:20.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.952+0000 7faa4bfff640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faa3806c8f0 0x7faa3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.953+0000 7faa4bfff640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faa3806c8f0 0x7faa3806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa40004f60 tx=0x7faa40007e30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:20.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:20.956+0000 7faa48ff9640 1 -- 172.21.15.37:0/3607768313 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa3c059690 con 0x7faa4c0d3820 2024-04-26T20:22:21.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.145+0000 7faa528cc640 1 -- 172.21.15.37:0/3607768313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa4c0d2960 con 0x7faa4c0d3820 2024-04-26T20:22:21.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.146+0000 7faa48ff9640 1 -- 172.21.15.37:0/3607768313 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7faa3c05d990 con 0x7faa4c0d3820 2024-04-26T20:22:21.147 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:21.147 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:21.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.148+0000 7faa528cc640 1 -- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faa3806c8f0 msgr2=0x7faa3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.148+0000 7faa528cc640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faa3806c8f0 0x7faa3806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa40004f60 tx=0x7faa40007e30 comp rx=0 tx=0).stop 2024-04-26T20:22:21.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.148+0000 7faa528cc640 1 -- 172.21.15.37:0/3607768313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa4c0d3820 msgr2=0x7faa4c0ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.148+0000 7faa528cc640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa4c0d3820 0x7faa4c0ca850 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00c820 tx=0x7faa3c00cce0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.148+0000 7faa528cc640 1 -- 172.21.15.37:0/3607768313 shutdown_connections 2024-04-26T20:22:21.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.148+0000 7faa528cc640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa4c162b40 0x7faa4c14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.148+0000 7faa528cc640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa4c0d3820 0x7faa4c0ca850 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.148+0000 7faa528cc640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faa3806c8f0 0x7faa3806edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.148+0000 7faa528cc640 1 --2- 172.21.15.37:0/3607768313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa4c0b4ba0 0x7faa4c0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.148+0000 7faa528cc640 1 -- 172.21.15.37:0/3607768313 >> 172.21.15.37:0/3607768313 conn(0x7faa4c09e550 msgr2=0x7faa4c0b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:21.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.149+0000 7faa528cc640 1 -- 172.21.15.37:0/3607768313 shutdown_connections 2024-04-26T20:22:21.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.149+0000 7faa528cc640 1 -- 172.21.15.37:0/3607768313 wait complete. 2024-04-26T20:22:21.161 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:21.161 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:21.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.264+0000 7fbc1dec8640 1 Processor -- start 2024-04-26T20:22:21.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.264+0000 7fbc1dec8640 1 -- start start 2024-04-26T20:22:21.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.265+0000 7fbc1dec8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d21b0 0x7fbc1816a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.265+0000 7fbc1dec8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc180d2ed0 con 0x7fbc1815f470 2024-04-26T20:22:21.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.265+0000 7fbc1dec8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc180d31b0 con 0x7fbc180d0f00 2024-04-26T20:22:21.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.265+0000 7fbc1dec8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbc180d3490 con 0x7fbc180d21b0 2024-04-26T20:22:21.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.265+0000 7fbc177fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbc1815f470 0x7fbc180d07f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32894/0 (socket says 172.21.15.37:32894) 2024-04-26T20:22:21.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.265+0000 7fbc177fe640 1 -- 172.21.15.37:0/2455339691 learned_addr learned my addr 172.21.15.37:0/2455339691 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:21.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.265+0000 7fbc17fff640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d21b0 0x7fbc1816a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.265+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 206857037 0 0) 0x7fbc180d2ed0 con 0x7fbc1815f470 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.265+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc00003520 con 0x7fbc1815f470 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 422945054 0 0) 0x7fbc00003520 con 0x7fbc1815f470 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d21b0 msgr2=0x7fbc1816a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d21b0 0x7fbc1816a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 >> v1:172.21.15.134:6789/0 conn(0x7fbc180d0f00 legacy=0x7fbc180d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc180d3770 con 0x7fbc1815f470 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fbc08001670 con 0x7fbc1815f470 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 >> v1:172.21.15.37:6789/0 conn(0x7fbc1815f470 legacy=0x7fbc180d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d21b0 0x7fbc000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 0x7fbc00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc00004d40 0x7fbc000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180d3490 con 0x7fbc00003f20 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0000ba20 con 0x7fbc00004d40 2024-04-26T20:22:21.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0000bc20 con 0x7fbc180d21b0 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc16ffd640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 0x7fbc00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc177fe640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d21b0 0x7fbc000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc17fff640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc00004d40 0x7fbc000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc16ffd640 1 -- 172.21.15.37:0/2455339691 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d21b0 msgr2=0x7fbc000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc16ffd640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d21b0 0x7fbc000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc16ffd640 1 -- 172.21.15.37:0/2455339691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc00004d40 msgr2=0x7fbc000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc16ffd640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc00004d40 0x7fbc000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.266+0000 7fbc16ffd640 1 -- 172.21.15.37:0/2455339691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc00003520 con 0x7fbc00003f20 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.267+0000 7fbc16ffd640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 0x7fbc00004800 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c001860 tx=0x7fbc00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efa7ac4a13bb3f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.267+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c037070 con 0x7fbc00003f20 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.267+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc0c02ab60 con 0x7fbc00003f20 2024-04-26T20:22:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.267+0000 7fbc167fc640 1 -- 172.21.15.37:0/2455339691 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c001430 con 0x7fbc00003f20 2024-04-26T20:22:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.268+0000 7fbc1dec8640 1 -- 172.21.15.37:0/2455339691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 msgr2=0x7fbc00004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.268+0000 7fbc1dec8640 1 --2- 172.21.15.37:0/2455339691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 0x7fbc00004800 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c001860 tx=0x7fbc00005830 comp rx=0 tx=0).stop 2024-04-26T20:22:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.268+0000 7fbc17fff640 1 -- 172.21.15.37:0/2455339691 reap_dead start 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.268+0000 7fbc1dec8640 1 -- 172.21.15.37:0/2455339691 shutdown_connections 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.268+0000 7fbc1dec8640 1 -- 172.21.15.37:0/2455339691 >> 172.21.15.37:0/2455339691 conn(0x7fbc1809e510 msgr2=0x7fbc180d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.268+0000 7fbc1dec8640 1 -- 172.21.15.37:0/2455339691 shutdown_connections 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.268+0000 7fbc1dec8640 1 -- 172.21.15.37:0/2455339691 wait complete. 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.269+0000 7fbc1dec8640 1 Processor -- start 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.269+0000 7fbc1dec8640 1 -- start start 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc1dec8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 0x7fbc180cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc1dec8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc181528f0 0x7fbc180cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc1dec8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc180cb700 0x7fbc180cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc1dec8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180d3f70 con 0x7fbc00003f20 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc1dec8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1815f850 con 0x7fbc180cb700 2024-04-26T20:22:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc1dec8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbc180ca400 con 0x7fbc181528f0 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 0x7fbc180cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 0x7fbc180cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60430/0 (socket says 172.21.15.37:60430) 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc177fe640 1 -- 172.21.15.37:0/3828268409 learned_addr learned my addr 172.21.15.37:0/3828268409 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc177fe640 1 -- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc181528f0 msgr2=0x7fbc180cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc16ffd640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc181528f0 0x7fbc180cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc17fff640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc180cb700 0x7fbc180cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc177fe640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc181528f0 0x7fbc180cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc177fe640 1 -- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc180cb700 msgr2=0x7fbc180cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc177fe640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc180cb700 0x7fbc180cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc177fe640 1 -- 172.21.15.37:0/3828268409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc180ca6e0 con 0x7fbc00003f20 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.270+0000 7fbc177fe640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 0x7fbc180cb1c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc080072c0 tx=0x7fbc08007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.271+0000 7fbbfffff640 1 -- 172.21.15.37:0/3828268409 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800f960 con 0x7fbc00003f20 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.271+0000 7fbbfffff640 1 -- 172.21.15.37:0/3828268409 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc08018070 con 0x7fbc00003f20 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.271+0000 7fbbfffff640 1 -- 172.21.15.37:0/3828268409 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800fd60 con 0x7fbc00003f20 2024-04-26T20:22:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.271+0000 7fbc1dec8640 1 -- 172.21.15.37:0/3828268409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc180ca9c0 con 0x7fbc00003f20 2024-04-26T20:22:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.271+0000 7fbc1dec8640 1 -- 172.21.15.37:0/3828268409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc180cd430 con 0x7fbc00003f20 2024-04-26T20:22:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.272+0000 7fbc1dec8640 1 -- 172.21.15.37:0/3828268409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1806a360 con 0x7fbc00003f20 2024-04-26T20:22:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.272+0000 7fbbfffff640 1 -- 172.21.15.37:0/3828268409 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc0801d460 con 0x7fbc00003f20 2024-04-26T20:22:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.272+0000 7fbbfffff640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc0406c920 0x7fbc0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.273+0000 7fbbfffff640 1 -- 172.21.15.37:0/3828268409 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbc0808bb00 con 0x7fbc00003f20 2024-04-26T20:22:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.273+0000 7fbc16ffd640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc0406c920 0x7fbc0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.273+0000 7fbc16ffd640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc0406c920 0x7fbc0406ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c034000 tx=0x7fbc0c009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.277+0000 7fbbfffff640 1 -- 172.21.15.37:0/3828268409 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc08059fc0 con 0x7fbc00003f20 2024-04-26T20:22:21.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.460+0000 7fbc1dec8640 1 -- 172.21.15.37:0/3828268409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbc180d5870 con 0x7fbc00003f20 2024-04-26T20:22:21.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.461+0000 7fbbfffff640 1 -- 172.21.15.37:0/3828268409 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fbc080598c0 con 0x7fbc00003f20 2024-04-26T20:22:21.462 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:21.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 -- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc0406c920 msgr2=0x7fbc0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc0406c920 0x7fbc0406ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c034000 tx=0x7fbc0c009030 comp rx=0 tx=0).stop 2024-04-26T20:22:21.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 -- 172.21.15.37:0/3828268409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 msgr2=0x7fbc180cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 0x7fbc180cb1c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc080072c0 tx=0x7fbc08007780 comp rx=0 tx=0).stop 2024-04-26T20:22:21.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 -- 172.21.15.37:0/3828268409 shutdown_connections 2024-04-26T20:22:21.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc180cb700 0x7fbc180cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbc0406c920 0x7fbc0406ede0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc181528f0 0x7fbc180cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 --2- 172.21.15.37:0/3828268409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc00003f20 0x7fbc180cb1c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 -- 172.21.15.37:0/3828268409 >> 172.21.15.37:0/3828268409 conn(0x7fbc1809e510 msgr2=0x7fbc18075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:21.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 -- 172.21.15.37:0/3828268409 shutdown_connections 2024-04-26T20:22:21.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.463+0000 7fbc1dec8640 1 -- 172.21.15.37:0/3828268409 wait complete. 2024-04-26T20:22:21.475 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.577+0000 7f21f63af640 1 Processor -- start 2024-04-26T20:22:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.577+0000 7f21f63af640 1 -- start start 2024-04-26T20:22:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.578+0000 7f21f63af640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f21f00d2320 0x7f21f00d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.578+0000 7f21f63af640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f0161510 con 0x7f21f0160800 2024-04-26T20:22:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.578+0000 7f21f63af640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f00d0930 con 0x7f21f01610c0 2024-04-26T20:22:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.578+0000 7f21f63af640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f21f00d0c00 con 0x7f21f00d2320 2024-04-26T20:22:21.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.578+0000 7f21effff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f21f0160800 0x7f21f00d5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32900/0 (socket says 172.21.15.37:32900) 2024-04-26T20:22:21.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.578+0000 7f21effff640 1 -- 172.21.15.37:0/1532436759 learned_addr learned my addr 172.21.15.37:0/1532436759 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:21.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.578+0000 7f21f494c640 1 --2- 172.21.15.37:0/1532436759 >> v2:172.21.15.134:3300/0 conn(0x7f21f00d2320 0x7f21f00d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2327895598 0 0) 0x7f21f0161510 con 0x7f21f0160800 2024-04-26T20:22:21.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21dc003520 con 0x7f21f0160800 2024-04-26T20:22:21.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2724073199 0 0) 0x7f21dc003520 con 0x7f21f0160800 2024-04-26T20:22:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 >> v2:172.21.15.134:3300/0 conn(0x7f21f00d2320 msgr2=0x7f21f00d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 --2- 172.21.15.37:0/1532436759 >> v2:172.21.15.134:3300/0 conn(0x7f21f00d2320 0x7f21f00d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 >> v1:172.21.15.134:6789/0 conn(0x7f21f01610c0 legacy=0x7f21f00d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f00d0ee0 con 0x7f21f0160800 2024-04-26T20:22:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f21d8001670 con 0x7f21f0160800 2024-04-26T20:22:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 >> v1:172.21.15.37:6789/0 conn(0x7f21f0160800 legacy=0x7f21f00d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 --2- 172.21.15.37:0/1532436759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21dc0039b0 0x7f21dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 --2- 172.21.15.37:0/1532436759 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21dc004800 0x7f21dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 --2- 172.21.15.37:0/1532436759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 0x7f21dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f21f00d0c00 con 0x7f21dc0051c0 2024-04-26T20:22:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc00c160 con 0x7f21dc0039b0 2024-04-26T20:22:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f21dc00c360 con 0x7f21dc004800 2024-04-26T20:22:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21f494c640 1 --2- 172.21.15.37:0/1532436759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 0x7f21dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21effff640 1 --2- 172.21.15.37:0/1532436759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21dc0039b0 0x7f21dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21ef7fe640 1 --2- 172.21.15.37:0/1532436759 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21dc004800 0x7f21dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21f494c640 1 -- 172.21.15.37:0/1532436759 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21dc004800 msgr2=0x7f21dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21f494c640 1 --2- 172.21.15.37:0/1532436759 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21dc004800 0x7f21dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21f494c640 1 -- 172.21.15.37:0/1532436759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21dc0039b0 msgr2=0x7f21dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21f494c640 1 --2- 172.21.15.37:0/1532436759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21dc0039b0 0x7f21dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.579+0000 7f21f494c640 1 -- 172.21.15.37:0/1532436759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc003520 con 0x7f21dc0051c0 2024-04-26T20:22:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.580+0000 7f21f494c640 1 -- 172.21.15.37:0/1532436759 reap_dead start 2024-04-26T20:22:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.580+0000 7f21f494c640 1 --2- 172.21.15.37:0/1532436759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 0x7f21dc00b9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009fc0 tx=0x7f21dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f89fa61f9034acd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.580+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8035020 con 0x7f21dc0051c0 2024-04-26T20:22:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.580+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e8003bd0 con 0x7f21dc0051c0 2024-04-26T20:22:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.580+0000 7f21eeffd640 1 -- 172.21.15.37:0/1532436759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8009c10 con 0x7f21dc0051c0 2024-04-26T20:22:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.581+0000 7f21f63af640 1 -- 172.21.15.37:0/1532436759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 msgr2=0x7f21dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.581+0000 7f21f63af640 1 --2- 172.21.15.37:0/1532436759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 0x7f21dc00b9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009fc0 tx=0x7f21dc005830 comp rx=0 tx=0).stop 2024-04-26T20:22:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.581+0000 7f21f63af640 1 -- 172.21.15.37:0/1532436759 shutdown_connections 2024-04-26T20:22:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.581+0000 7f21f63af640 1 --2- 172.21.15.37:0/1532436759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 0x7f21dc00b9c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.581+0000 7f21f63af640 1 -- 172.21.15.37:0/1532436759 >> 172.21.15.37:0/1532436759 conn(0x7f21f009e550 msgr2=0x7f21f00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.581+0000 7f21f63af640 1 -- 172.21.15.37:0/1532436759 shutdown_connections 2024-04-26T20:22:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.581+0000 7f21f63af640 1 -- 172.21.15.37:0/1532436759 wait complete. 2024-04-26T20:22:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.582+0000 7f21f63af640 1 Processor -- start 2024-04-26T20:22:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.582+0000 7f21f63af640 1 -- start start 2024-04-26T20:22:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21f63af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 0x7f21f015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21f63af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f00b8710 0x7f21f00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21f63af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21f00ce690 0x7f21f00ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21f63af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f21f00d30f0 con 0x7f21dc0051c0 2024-04-26T20:22:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21f63af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21f00d2e10 con 0x7f21f00b8710 2024-04-26T20:22:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21f63af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f21f00b6300 con 0x7f21f00ce690 2024-04-26T20:22:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21effff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 0x7f21f015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21effff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 0x7f21f015ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60440/0 (socket says 172.21.15.37:60440) 2024-04-26T20:22:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21effff640 1 -- 172.21.15.37:0/3160353473 learned_addr learned my addr 172.21.15.37:0/3160353473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21ef7fe640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f00b8710 0x7f21f00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21f494c640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21f00ce690 0x7f21f00ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21effff640 1 -- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21f00ce690 msgr2=0x7f21f00ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21effff640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21f00ce690 0x7f21f00ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21effff640 1 -- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f00b8710 msgr2=0x7f21f00cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21effff640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f00b8710 0x7f21f00cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21effff640 1 -- 172.21.15.37:0/3160353473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f015d350 con 0x7f21dc0051c0 2024-04-26T20:22:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.583+0000 7f21effff640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 0x7f21f015ce10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f21d80072c0 tx=0x7f21d8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.584+0000 7f21ecff9640 1 -- 172.21.15.37:0/3160353473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80016c0 con 0x7f21dc0051c0 2024-04-26T20:22:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.584+0000 7f21ecff9640 1 -- 172.21.15.37:0/3160353473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d8018070 con 0x7f21dc0051c0 2024-04-26T20:22:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.584+0000 7f21f63af640 1 -- 172.21.15.37:0/3160353473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f00b2e40 con 0x7f21dc0051c0 2024-04-26T20:22:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.584+0000 7f21ecff9640 1 -- 172.21.15.37:0/3160353473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d800fc00 con 0x7f21dc0051c0 2024-04-26T20:22:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.584+0000 7f21f63af640 1 -- 172.21.15.37:0/3160353473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f00b8e80 con 0x7f21dc0051c0 2024-04-26T20:22:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.585+0000 7f21f63af640 1 -- 172.21.15.37:0/3160353473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f006a360 con 0x7f21dc0051c0 2024-04-26T20:22:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.585+0000 7f21ecff9640 1 -- 172.21.15.37:0/3160353473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21d801d460 con 0x7f21dc0051c0 2024-04-26T20:22:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.586+0000 7f21ecff9640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f21d006c8f0 0x7f21d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.586+0000 7f21ecff9640 1 -- 172.21.15.37:0/3160353473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f21d808bb80 con 0x7f21dc0051c0 2024-04-26T20:22:21.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.586+0000 7f21ef7fe640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f21d006c8f0 0x7f21d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.586+0000 7f21ef7fe640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f21d006c8f0 0x7f21d006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f21f0074680 tx=0x7f21e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:21.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.590+0000 7f21ecff9640 1 -- 172.21.15.37:0/3160353473 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f21f00b2e40 con 0x7f21dc0051c0 2024-04-26T20:22:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.775+0000 7f21f63af640 1 -- 172.21.15.37:0/3160353473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f21f00d4530 con 0x7f21dc0051c0 2024-04-26T20:22:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.775+0000 7f21ecff9640 1 -- 172.21.15.37:0/3160353473 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f21d805a040 con 0x7f21dc0051c0 2024-04-26T20:22:21.777 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:21.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 -- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f21d006c8f0 msgr2=0x7f21d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f21d006c8f0 0x7f21d006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f21f0074680 tx=0x7f21e0009040 comp rx=0 tx=0).stop 2024-04-26T20:22:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 -- 172.21.15.37:0/3160353473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 msgr2=0x7f21f015ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 0x7f21f015ce10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f21d80072c0 tx=0x7f21d8007780 comp rx=0 tx=0).stop 2024-04-26T20:22:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 -- 172.21.15.37:0/3160353473 shutdown_connections 2024-04-26T20:22:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f21d006c8f0 0x7f21d006edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21f00ce690 0x7f21f00ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f00b8710 0x7f21f00cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 --2- 172.21.15.37:0/3160353473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21dc0051c0 0x7f21f015ce10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 -- 172.21.15.37:0/3160353473 >> 172.21.15.37:0/3160353473 conn(0x7f21f009e550 msgr2=0x7f21f015e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 -- 172.21.15.37:0/3160353473 shutdown_connections 2024-04-26T20:22:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.777+0000 7f21f63af640 1 -- 172.21.15.37:0/3160353473 wait complete. 2024-04-26T20:22:21.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.892+0000 7f35d9293640 1 Processor -- start 2024-04-26T20:22:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.892+0000 7f35d9293640 1 -- start start 2024-04-26T20:22:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.893+0000 7f35d9293640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f35d415f5e0 0x7f35d406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.893+0000 7f35d9293640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35d40d2b30 0x7f35d406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.893+0000 7f35d9293640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d40d25f0 con 0x7f35d406d560 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.893+0000 7f35d9293640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f35d40cda70 con 0x7f35d40d2b30 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.893+0000 7f35d9293640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f35d40d2030 con 0x7f35d415f5e0 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.893+0000 7f35d27fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35d40d2b30 0x7f35d406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.893+0000 7f35d27fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35d40d2b30 0x7f35d406e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60442/0 (socket says 172.21.15.37:60442) 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.893+0000 7f35d27fc640 1 -- 172.21.15.37:0/283398069 learned_addr learned my addr 172.21.15.37:0/283398069 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.893+0000 7f35d2ffd640 1 --2- 172.21.15.37:0/283398069 >> v2:172.21.15.134:3300/0 conn(0x7f35d415f5e0 0x7f35d406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.894+0000 7f35d27fc640 1 -- 172.21.15.37:0/283398069 >> v2:172.21.15.134:3300/0 conn(0x7f35d415f5e0 msgr2=0x7f35d406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.894+0000 7f35d27fc640 1 --2- 172.21.15.37:0/283398069 >> v2:172.21.15.134:3300/0 conn(0x7f35d415f5e0 0x7f35d406dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.894+0000 7f35d27fc640 1 -- 172.21.15.37:0/283398069 >> v1:172.21.15.134:6789/0 conn(0x7f35d406d560 legacy=0x7f35d40cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.894+0000 7f35d27fc640 1 -- 172.21.15.37:0/283398069 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d40d2310 con 0x7f35d40d2b30 2024-04-26T20:22:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.894+0000 7f35d27fc640 1 --2- 172.21.15.37:0/283398069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40d2b30 0x7f35d406e2a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f35d4070210 tx=0x7f35c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b95d94e07baa74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.895+0000 7f35d1ffb640 1 -- 172.21.15.37:0/283398069 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8037070 con 0x7f35d40d2b30 2024-04-26T20:22:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.895+0000 7f35d1ffb640 1 -- 172.21.15.37:0/283398069 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c8032440 con 0x7f35d40d2b30 2024-04-26T20:22:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.895+0000 7f35d1ffb640 1 -- 172.21.15.37:0/283398069 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8032810 con 0x7f35d40d2b30 2024-04-26T20:22:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.895+0000 7f35d9293640 1 -- 172.21.15.37:0/283398069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40d2b30 msgr2=0x7f35d406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.895+0000 7f35d9293640 1 --2- 172.21.15.37:0/283398069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40d2b30 0x7f35d406e2a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f35d4070210 tx=0x7f35c802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.895+0000 7f35d9293640 1 -- 172.21.15.37:0/283398069 shutdown_connections 2024-04-26T20:22:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.895+0000 7f35d9293640 1 --2- 172.21.15.37:0/283398069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d40d2b30 0x7f35d406e2a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.895+0000 7f35d9293640 1 --2- 172.21.15.37:0/283398069 >> v2:172.21.15.134:3300/0 conn(0x7f35d415f5e0 0x7f35d406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.895+0000 7f35d9293640 1 -- 172.21.15.37:0/283398069 >> 172.21.15.37:0/283398069 conn(0x7f35d40d2f80 msgr2=0x7f35d40d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.895+0000 7f35d9293640 1 -- 172.21.15.37:0/283398069 shutdown_connections 2024-04-26T20:22:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.895+0000 7f35d9293640 1 -- 172.21.15.37:0/283398069 wait complete. 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.897+0000 7f35d9293640 1 Processor -- start 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.897+0000 7f35d9293640 1 -- start start 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d9293640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d406d560 0x7f35d415ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d9293640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d40d2b30 0x7f35d415d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d9293640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35d415f5e0 0x7f35d415d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d9293640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f35d40cef10 con 0x7f35d406d560 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d9293640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35d40d1cb0 con 0x7f35d40d2b30 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d9293640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f35d40cf090 con 0x7f35d415f5e0 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d406d560 0x7f35d415ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d406d560 0x7f35d415ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60450/0 (socket says 172.21.15.37:60450) 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d2ffd640 1 -- 172.21.15.37:0/522995676 learned_addr learned my addr 172.21.15.37:0/522995676 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d27fc640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d40d2b30 0x7f35d415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d37fe640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35d415f5e0 0x7f35d415d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d2ffd640 1 -- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35d415f5e0 msgr2=0x7f35d415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d2ffd640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35d415f5e0 0x7f35d415d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d2ffd640 1 -- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d40d2b30 msgr2=0x7f35d415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d2ffd640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d40d2b30 0x7f35d415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d2ffd640 1 -- 172.21.15.37:0/522995676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d41f8ef0 con 0x7f35d406d560 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d2ffd640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d406d560 0x7f35d415ff80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e9e0 tx=0x7f35c400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35bb7fe640 1 -- 172.21.15.37:0/522995676 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4017070 con 0x7f35d406d560 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35bb7fe640 1 -- 172.21.15.37:0/522995676 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c400c9d0 con 0x7f35d406d560 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.898+0000 7f35d9293640 1 -- 172.21.15.37:0/522995676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d41f91d0 con 0x7f35d406d560 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.899+0000 7f35bb7fe640 1 -- 172.21.15.37:0/522995676 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c400cdc0 con 0x7f35d406d560 2024-04-26T20:22:21.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.899+0000 7f35d9293640 1 -- 172.21.15.37:0/522995676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d41604c0 con 0x7f35d406d560 2024-04-26T20:22:21.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.900+0000 7f35d9293640 1 -- 172.21.15.37:0/522995676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d40693e0 con 0x7f35d406d560 2024-04-26T20:22:21.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.900+0000 7f35bb7fe640 1 -- 172.21.15.37:0/522995676 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35c401a070 con 0x7f35d406d560 2024-04-26T20:22:21.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.900+0000 7f35bb7fe640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35c006c8f0 0x7f35c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:21.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.901+0000 7f35bb7fe640 1 -- 172.21.15.37:0/522995676 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f35c408be40 con 0x7f35d406d560 2024-04-26T20:22:21.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.901+0000 7f35d27fc640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35c006c8f0 0x7f35c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.901+0000 7f35d27fc640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35c006c8f0 0x7f35c006edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f35c8034000 tx=0x7f35c802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:21.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:21.905+0000 7f35bb7fe640 1 -- 172.21.15.37:0/522995676 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f35c4059c20 con 0x7f35d406d560 2024-04-26T20:22:22.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.088+0000 7f35d9293640 1 -- 172.21.15.37:0/522995676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f35d40c6860 con 0x7f35d406d560 2024-04-26T20:22:22.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.089+0000 7f35bb7fe640 1 -- 172.21.15.37:0/522995676 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f35c405df20 con 0x7f35d406d560 2024-04-26T20:22:22.090 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:22.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 -- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35c006c8f0 msgr2=0x7f35c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35c006c8f0 0x7f35c006edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f35c8034000 tx=0x7f35c802e000 comp rx=0 tx=0).stop 2024-04-26T20:22:22.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 -- 172.21.15.37:0/522995676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d406d560 msgr2=0x7f35d415ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d406d560 0x7f35d415ff80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e9e0 tx=0x7f35c400eea0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 -- 172.21.15.37:0/522995676 shutdown_connections 2024-04-26T20:22:22.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f35c006c8f0 0x7f35c006edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35d415f5e0 0x7f35d415d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d40d2b30 0x7f35d415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 --2- 172.21.15.37:0/522995676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d406d560 0x7f35d415ff80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 -- 172.21.15.37:0/522995676 >> 172.21.15.37:0/522995676 conn(0x7f35d40d2f80 msgr2=0x7f35d40b8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:22.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 -- 172.21.15.37:0/522995676 shutdown_connections 2024-04-26T20:22:22.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.091+0000 7f35d9293640 1 -- 172.21.15.37:0/522995676 wait complete. 2024-04-26T20:22:22.103 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:22:22.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.203+0000 7fab7c68c640 1 Processor -- start 2024-04-26T20:22:22.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.203+0000 7fab7c68c640 1 -- start start 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7c68c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fab74162e40 0x7fab740d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7c68c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab740b2050 0x7fab740cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7c68c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fab740cfd70 0x7fab740ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7c68c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fab74163220 con 0x7fab74162e40 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7c68c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fab740d0eb0 con 0x7fab740cfd70 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7c68c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fab740d01a0 con 0x7fab740b2050 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7a428640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fab74162e40 0x7fab740d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7a428640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fab74162e40 0x7fab740d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60466/0 (socket says 172.21.15.37:60466) 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7a428640 1 -- 172.21.15.37:0/4243187249 learned_addr learned my addr 172.21.15.37:0/4243187249 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab79c27640 1 --2- 172.21.15.37:0/4243187249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab740b2050 0x7fab740cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7ac29640 1 --2- 172.21.15.37:0/4243187249 >> v2:172.21.15.134:3300/0 conn(0x7fab740cfd70 0x7fab740ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7a428640 1 -- 172.21.15.37:0/4243187249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab740b2050 msgr2=0x7fab740cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:22.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7a428640 1 --2- 172.21.15.37:0/4243187249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab740b2050 0x7fab740cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7a428640 1 -- 172.21.15.37:0/4243187249 >> v2:172.21.15.134:3300/0 conn(0x7fab740cfd70 msgr2=0x7fab740ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:22.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7a428640 1 --2- 172.21.15.37:0/4243187249 >> v2:172.21.15.134:3300/0 conn(0x7fab740cfd70 0x7fab740ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7a428640 1 -- 172.21.15.37:0/4243187249 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab740b1d70 con 0x7fab74162e40 2024-04-26T20:22:22.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab79c27640 1 --2- 172.21.15.37:0/4243187249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab740b2050 0x7fab740cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:22.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.204+0000 7fab7ac29640 1 --2- 172.21.15.37:0/4243187249 >> v2:172.21.15.134:3300/0 conn(0x7fab740cfd70 0x7fab740ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:22.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.205+0000 7fab7a428640 1 --2- 172.21.15.37:0/4243187249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab74162e40 0x7fab740d1340 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab74072bb0 tx=0x7fab6402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9044d41eed6560 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:22.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.205+0000 7fab79426640 1 -- 172.21.15.37:0/4243187249 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64036070 con 0x7fab74162e40 2024-04-26T20:22:22.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.205+0000 7fab79426640 1 -- 172.21.15.37:0/4243187249 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab64031600 con 0x7fab74162e40 2024-04-26T20:22:22.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.205+0000 7fab79426640 1 -- 172.21.15.37:0/4243187249 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab640319d0 con 0x7fab74162e40 2024-04-26T20:22:22.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.206+0000 7fab7c68c640 1 -- 172.21.15.37:0/4243187249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab74162e40 msgr2=0x7fab740d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.206+0000 7fab7c68c640 1 --2- 172.21.15.37:0/4243187249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab74162e40 0x7fab740d1340 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab74072bb0 tx=0x7fab6402a610 comp rx=0 tx=0).stop 2024-04-26T20:22:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.206+0000 7fab7c68c640 1 -- 172.21.15.37:0/4243187249 shutdown_connections 2024-04-26T20:22:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.206+0000 7fab7c68c640 1 --2- 172.21.15.37:0/4243187249 >> v2:172.21.15.134:3300/0 conn(0x7fab740cfd70 0x7fab740ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.206+0000 7fab7c68c640 1 --2- 172.21.15.37:0/4243187249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab740b2050 0x7fab740cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.206+0000 7fab7c68c640 1 --2- 172.21.15.37:0/4243187249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab74162e40 0x7fab740d1340 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.206+0000 7fab7c68c640 1 -- 172.21.15.37:0/4243187249 >> 172.21.15.37:0/4243187249 conn(0x7fab7409ba50 msgr2=0x7fab7409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.206+0000 7fab7c68c640 1 -- 172.21.15.37:0/4243187249 shutdown_connections 2024-04-26T20:22:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.206+0000 7fab7c68c640 1 -- 172.21.15.37:0/4243187249 wait complete. 2024-04-26T20:22:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.207+0000 7fab7c68c640 1 Processor -- start 2024-04-26T20:22:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.207+0000 7fab7c68c640 1 -- start start 2024-04-26T20:22:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7c68c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab740b2050 0x7fab7415c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7c68c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab740cfd70 0x7fab740b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7c68c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab74162e40 0x7fab740b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7c68c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab740d1f20 con 0x7fab74162e40 2024-04-26T20:22:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7c68c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab740d1c40 con 0x7fab740cfd70 2024-04-26T20:22:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7c68c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fab740b24b0 con 0x7fab740b2050 2024-04-26T20:22:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7ac29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab74162e40 0x7fab740b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7ac29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab74162e40 0x7fab740b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60482/0 (socket says 172.21.15.37:60482) 2024-04-26T20:22:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7ac29640 1 -- 172.21.15.37:0/2464636313 learned_addr learned my addr 172.21.15.37:0/2464636313 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab79c27640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab740cfd70 0x7fab740b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7ac29640 1 -- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab740b2050 msgr2=0x7fab7415c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7ac29640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab740b2050 0x7fab7415c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7ac29640 1 -- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab740cfd70 msgr2=0x7fab740b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7ac29640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab740cfd70 0x7fab740b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7ac29640 1 -- 172.21.15.37:0/2464636313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab740cb610 con 0x7fab74162e40 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.208+0000 7fab7ac29640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab74162e40 0x7fab740b3cf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00c7f0 tx=0x7fab6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.209+0000 7fab72ffd640 1 -- 172.21.15.37:0/2464636313 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c015070 con 0x7fab74162e40 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.209+0000 7fab72ffd640 1 -- 172.21.15.37:0/2464636313 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c007820 con 0x7fab74162e40 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.209+0000 7fab7c68c640 1 -- 172.21.15.37:0/2464636313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab740cb8f0 con 0x7fab74162e40 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.209+0000 7fab72ffd640 1 -- 172.21.15.37:0/2464636313 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c007c50 con 0x7fab74162e40 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.209+0000 7fab7c68c640 1 -- 172.21.15.37:0/2464636313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab74160f90 con 0x7fab74162e40 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.210+0000 7fab7c68c640 1 -- 172.21.15.37:0/2464636313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7406a360 con 0x7fab74162e40 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.210+0000 7fab72ffd640 1 -- 172.21.15.37:0/2464636313 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab6c018070 con 0x7fab74162e40 2024-04-26T20:22:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.211+0000 7fab72ffd640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab6006c8f0 0x7fab6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.211+0000 7fab72ffd640 1 -- 172.21.15.37:0/2464636313 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fab6c08bb20 con 0x7fab74162e40 2024-04-26T20:22:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.211+0000 7fab7a428640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab6006c8f0 0x7fab6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.211+0000 7fab7a428640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab6006c8f0 0x7fab6006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab64033000 tx=0x7fab6402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:22.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.215+0000 7fab72ffd640 1 -- 172.21.15.37:0/2464636313 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab6c059f50 con 0x7fab74162e40 2024-04-26T20:22:22.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.416+0000 7fab7c68c640 1 -- 172.21.15.37:0/2464636313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fab741f5f70 con 0x7fab74162e40 2024-04-26T20:22:22.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.416+0000 7fab72ffd640 1 -- 172.21.15.37:0/2464636313 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fab6c059850 con 0x7fab74162e40 2024-04-26T20:22:22.417 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-26T20:22:22.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 -- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab6006c8f0 msgr2=0x7fab6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab6006c8f0 0x7fab6006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab64033000 tx=0x7fab6402e000 comp rx=0 tx=0).stop 2024-04-26T20:22:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 -- 172.21.15.37:0/2464636313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab74162e40 msgr2=0x7fab740b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab74162e40 0x7fab740b3cf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00c7f0 tx=0x7fab6c00ccb0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 -- 172.21.15.37:0/2464636313 shutdown_connections 2024-04-26T20:22:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab74162e40 0x7fab740b3cf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab740cfd70 0x7fab740b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab6006c8f0 0x7fab6006edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 --2- 172.21.15.37:0/2464636313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab740b2050 0x7fab7415c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 -- 172.21.15.37:0/2464636313 >> 172.21.15.37:0/2464636313 conn(0x7fab7409ba50 msgr2=0x7fab7409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 -- 172.21.15.37:0/2464636313 shutdown_connections 2024-04-26T20:22:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.418+0000 7fab7c68c640 1 -- 172.21.15.37:0/2464636313 wait complete. 2024-04-26T20:22:22.430 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T20:22:22.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:22.430 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T20:22:22.456 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:22:22.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:22.456 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:22:22.517 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:22:22.517 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:22.535 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:22:22.535 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:22.593 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:22:22.593 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:22.650 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:22.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.750+0000 7f2a7bbf7640 1 Processor -- start 2024-04-26T20:22:22.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.750+0000 7f2a7bbf7640 1 -- start start 2024-04-26T20:22:22.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.751+0000 7f2a7bbf7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2a74162e40 0x7f2a740d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.751+0000 7f2a7bbf7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740cff40 0x7f2a740ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.751+0000 7f2a7bbf7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a740b1d70 con 0x7f2a740b2050 2024-04-26T20:22:22.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.751+0000 7f2a7bbf7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2a74163220 con 0x7f2a74162e40 2024-04-26T20:22:22.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.751+0000 7f2a7bbf7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2a740d0eb0 con 0x7f2a740cff40 2024-04-26T20:22:22.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.751+0000 7f2a79192640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2a740b2050 0x7f2a740cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32912/0 (socket says 172.21.15.37:32912) 2024-04-26T20:22:22.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.751+0000 7f2a79192640 1 -- 172.21.15.37:0/2802074249 learned_addr learned my addr 172.21.15.37:0/2802074249 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:22.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.751+0000 7f2a7a194640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740cff40 0x7f2a740ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.751+0000 7f2a79993640 1 --2- 172.21.15.37:0/2802074249 >> v2:172.21.15.134:3300/0 conn(0x7f2a74162e40 0x7f2a740d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.751+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3660340534 0 0) 0x7f2a740b1d70 con 0x7f2a740b2050 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.751+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a5c003520 con 0x7f2a740b2050 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1457978709 0 0) 0x7f2a5c003520 con 0x7f2a740b2050 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740cff40 msgr2=0x7f2a740ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740cff40 0x7f2a740ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 >> v2:172.21.15.134:3300/0 conn(0x7f2a74162e40 msgr2=0x7f2a740d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 --2- 172.21.15.37:0/2802074249 >> v2:172.21.15.134:3300/0 conn(0x7f2a74162e40 0x7f2a740d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a740d1960 con 0x7f2a740b2050 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f2a68001690 con 0x7f2a740b2050 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 >> v1:172.21.15.37:6789/0 conn(0x7f2a740b2050 legacy=0x7f2a740cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740cff40 0x7f2a5c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a5c003f30 0x7f2a5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a5c004d40 0x7f2a5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a740d0eb0 con 0x7f2a5c003f30 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c00b9c0 con 0x7f2a5c004d40 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5c00bb90 con 0x7f2a740cff40 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a79192640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a5c003f30 0x7f2a5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a79993640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740cff40 0x7f2a5c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a7a194640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a5c004d40 0x7f2a5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a79192640 1 -- 172.21.15.37:0/2802074249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740cff40 msgr2=0x7f2a5c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a79192640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740cff40 0x7f2a5c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a79192640 1 -- 172.21.15.37:0/2802074249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a5c004d40 msgr2=0x7f2a5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a79192640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a5c004d40 0x7f2a5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.752+0000 7f2a79192640 1 -- 172.21.15.37:0/2802074249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c003520 con 0x7f2a5c003f30 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.753+0000 7f2a79192640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a5c003f30 0x7f2a5c004800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c240 tx=0x7f2a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537d512242bf77b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:22.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.753+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68007d70 con 0x7f2a5c003f30 2024-04-26T20:22:22.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.753+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a68005af0 con 0x7f2a5c003f30 2024-04-26T20:22:22.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.753+0000 7f2a78991640 1 -- 172.21.15.37:0/2802074249 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68007070 con 0x7f2a5c003f30 2024-04-26T20:22:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.753+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/2802074249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a5c003f30 msgr2=0x7f2a5c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.753+0000 7f2a7bbf7640 1 --2- 172.21.15.37:0/2802074249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a5c003f30 0x7f2a5c004800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c240 tx=0x7f2a5c005830 comp rx=0 tx=0).stop 2024-04-26T20:22:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.754+0000 7f2a7a194640 1 -- 172.21.15.37:0/2802074249 reap_dead start 2024-04-26T20:22:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.754+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/2802074249 shutdown_connections 2024-04-26T20:22:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.754+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/2802074249 >> 172.21.15.37:0/2802074249 conn(0x7f2a7409ba50 msgr2=0x7f2a7409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.754+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/2802074249 shutdown_connections 2024-04-26T20:22:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.754+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/2802074249 wait complete. 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.755+0000 7f2a7bbf7640 1 Processor -- start 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.755+0000 7f2a7bbf7640 1 -- start start 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a7bbf7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740c9600 0x7f2a740cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a7bbf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a741f5950 0x7f2a74150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a7bbf7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a741511f0 0x7f2a74151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a7bbf7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a740b24e0 con 0x7f2a741f5950 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a7bbf7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a740d1f20 con 0x7f2a741511f0 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a7bbf7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7406ea40 con 0x7f2a740c9600 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a79192640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a741f5950 0x7f2a74150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a79192640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a741f5950 0x7f2a74150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60510/0 (socket says 172.21.15.37:60510) 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a79192640 1 -- 172.21.15.37:0/3150155253 learned_addr learned my addr 172.21.15.37:0/3150155253 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a79993640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740c9600 0x7f2a740cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a7a194640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a741511f0 0x7f2a74151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a79192640 1 -- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740c9600 msgr2=0x7f2a740cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a79192640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740c9600 0x7f2a740cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a79192640 1 -- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a741511f0 msgr2=0x7f2a74151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a79192640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a741511f0 0x7f2a74151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a79192640 1 -- 172.21.15.37:0/3150155253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a740cc610 con 0x7f2a741f5950 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a79993640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740c9600 0x7f2a740cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a79192640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a741f5950 0x7f2a74150cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2a68034040 tx=0x7f2a68001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a727fc640 1 -- 172.21.15.37:0/3150155253 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68042020 con 0x7f2a741f5950 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a727fc640 1 -- 172.21.15.37:0/3150155253 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a68004cb0 con 0x7f2a741f5950 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/3150155253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a740cc8f0 con 0x7f2a741f5950 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a727fc640 1 -- 172.21.15.37:0/3150155253 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68001cb0 con 0x7f2a741f5950 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.756+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/3150155253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a74160b60 con 0x7f2a741f5950 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.757+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/3150155253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7406a360 con 0x7f2a741f5950 2024-04-26T20:22:22.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.758+0000 7f2a727fc640 1 -- 172.21.15.37:0/3150155253 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a68045020 con 0x7f2a741f5950 2024-04-26T20:22:22.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.758+0000 7f2a727fc640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2a6006c920 0x7f2a6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:22.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.758+0000 7f2a727fc640 1 -- 172.21.15.37:0/3150155253 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2a680abef0 con 0x7f2a741f5950 2024-04-26T20:22:22.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.758+0000 7f2a79993640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2a6006c920 0x7f2a6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:22.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.759+0000 7f2a79993640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2a6006c920 0x7f2a6006ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2a640074d0 tx=0x7f2a6400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:22.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.762+0000 7f2a727fc640 1 -- 172.21.15.37:0/3150155253 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a6804c070 con 0x7f2a741f5950 2024-04-26T20:22:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.953+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/3150155253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a740cce10 con 0x7f2a741f5950 2024-04-26T20:22:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.954+0000 7f2a727fc640 1 -- 172.21.15.37:0/3150155253 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f2a6807a330 con 0x7f2a741f5950 2024-04-26T20:22:22.956 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:22.956 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2a6006c920 msgr2=0x7f2a6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2a6006c920 0x7f2a6006ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2a640074d0 tx=0x7f2a6400b040 comp rx=0 tx=0).stop 2024-04-26T20:22:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/3150155253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a741f5950 msgr2=0x7f2a74150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a741f5950 0x7f2a74150cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2a68034040 tx=0x7f2a68001a60 comp rx=0 tx=0).stop 2024-04-26T20:22:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/3150155253 shutdown_connections 2024-04-26T20:22:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a741511f0 0x7f2a74151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a741f5950 0x7f2a74150cb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2a6006c920 0x7f2a6006ede0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 --2- 172.21.15.37:0/3150155253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a740c9600 0x7f2a740cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/3150155253 >> 172.21.15.37:0/3150155253 conn(0x7f2a7409ba50 msgr2=0x7f2a740702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/3150155253 shutdown_connections 2024-04-26T20:22:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:22.957+0000 7f2a7bbf7640 1 -- 172.21.15.37:0/3150155253 wait complete. 2024-04-26T20:22:22.969 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:22.969 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:23.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.072+0000 7ff8624d1640 1 Processor -- start 2024-04-26T20:22:23.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.072+0000 7ff8624d1640 1 -- start start 2024-04-26T20:22:23.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.072+0000 7ff8624d1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff85c15ffc0 0x7ff85c0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.072+0000 7ff8624d1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff85c1603a0 0x7ff85c160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.072+0000 7ff8624d1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff85c0cbd60 0x7ff85c0cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.072+0000 7ff8624d1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff85c06f690 con 0x7ff85c15ffc0 2024-04-26T20:22:23.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.072+0000 7ff8624d1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff85c06f970 con 0x7ff85c1603a0 2024-04-26T20:22:23.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.072+0000 7ff8624d1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff85c0735a0 con 0x7ff85c0cbd60 2024-04-26T20:22:23.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.073+0000 7ff85bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff85c15ffc0 0x7ff85c0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.073+0000 7ff85bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff85c15ffc0 0x7ff85c0cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60512/0 (socket says 172.21.15.37:60512) 2024-04-26T20:22:23.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.073+0000 7ff85bfff640 1 -- 172.21.15.37:0/410489362 learned_addr learned my addr 172.21.15.37:0/410489362 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:23.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.073+0000 7ff860a6e640 1 --2- 172.21.15.37:0/410489362 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff85c0cbd60 0x7ff85c0cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.073+0000 7ff85b7fe640 1 --2- 172.21.15.37:0/410489362 >> v2:172.21.15.134:3300/0 conn(0x7ff85c1603a0 0x7ff85c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.073+0000 7ff85bfff640 1 -- 172.21.15.37:0/410489362 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff85c0cbd60 msgr2=0x7ff85c0cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:23.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.073+0000 7ff85bfff640 1 --2- 172.21.15.37:0/410489362 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff85c0cbd60 0x7ff85c0cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.073+0000 7ff85bfff640 1 -- 172.21.15.37:0/410489362 >> v2:172.21.15.134:3300/0 conn(0x7ff85c1603a0 msgr2=0x7ff85c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:23.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.073+0000 7ff85bfff640 1 --2- 172.21.15.37:0/410489362 >> v2:172.21.15.134:3300/0 conn(0x7ff85c1603a0 0x7ff85c160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.073+0000 7ff85bfff640 1 -- 172.21.15.37:0/410489362 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c073880 con 0x7ff85c15ffc0 2024-04-26T20:22:23.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.073+0000 7ff85b7fe640 1 --2- 172.21.15.37:0/410489362 >> v2:172.21.15.134:3300/0 conn(0x7ff85c1603a0 0x7ff85c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:23.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff85bfff640 1 --2- 172.21.15.37:0/410489362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff85c15ffc0 0x7ff85c0cb7f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff85c09f0d0 tx=0x7ff84402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367f7fcbff41102b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:23.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff85affd640 1 -- 172.21.15.37:0/410489362 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844037070 con 0x7ff85c15ffc0 2024-04-26T20:22:23.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff85affd640 1 -- 172.21.15.37:0/410489362 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff844032440 con 0x7ff85c15ffc0 2024-04-26T20:22:23.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff85affd640 1 -- 172.21.15.37:0/410489362 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844032810 con 0x7ff85c15ffc0 2024-04-26T20:22:23.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff8624d1640 1 -- 172.21.15.37:0/410489362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff85c15ffc0 msgr2=0x7ff85c0cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff8624d1640 1 --2- 172.21.15.37:0/410489362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff85c15ffc0 0x7ff85c0cb7f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff85c09f0d0 tx=0x7ff84402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff8624d1640 1 -- 172.21.15.37:0/410489362 shutdown_connections 2024-04-26T20:22:23.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff8624d1640 1 --2- 172.21.15.37:0/410489362 >> v2:172.21.15.134:3300/0 conn(0x7ff85c1603a0 0x7ff85c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff8624d1640 1 --2- 172.21.15.37:0/410489362 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff85c0cbd60 0x7ff85c0cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff8624d1640 1 --2- 172.21.15.37:0/410489362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff85c15ffc0 0x7ff85c0cb7f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff8624d1640 1 -- 172.21.15.37:0/410489362 >> 172.21.15.37:0/410489362 conn(0x7ff85c0d0d70 msgr2=0x7ff85c0d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:23.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff8624d1640 1 -- 172.21.15.37:0/410489362 shutdown_connections 2024-04-26T20:22:23.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.074+0000 7ff8624d1640 1 -- 172.21.15.37:0/410489362 wait complete. 2024-04-26T20:22:23.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.075+0000 7ff8624d1640 1 Processor -- start 2024-04-26T20:22:23.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.076+0000 7ff8624d1640 1 -- start start 2024-04-26T20:22:23.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.076+0000 7ff8624d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff85c0cbd60 0x7ff85c09e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.076+0000 7ff8624d1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff85c15ffc0 0x7ff85c0d0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.076+0000 7ff8624d1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff85c1603a0 0x7ff85c157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.076+0000 7ff8624d1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c160d60 con 0x7ff85c0cbd60 2024-04-26T20:22:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.076+0000 7ff8624d1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c06fc50 con 0x7ff85c1603a0 2024-04-26T20:22:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.076+0000 7ff8624d1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff85c160ee0 con 0x7ff85c15ffc0 2024-04-26T20:22:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.076+0000 7ff85bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff85c0cbd60 0x7ff85c09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.076+0000 7ff85bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff85c0cbd60 0x7ff85c09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60520/0 (socket says 172.21.15.37:60520) 2024-04-26T20:22:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff85bfff640 1 -- 172.21.15.37:0/1833128601 learned_addr learned my addr 172.21.15.37:0/1833128601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff85b7fe640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff85c15ffc0 0x7ff85c0d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff85bfff640 1 -- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff85c15ffc0 msgr2=0x7ff85c0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff85bfff640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff85c15ffc0 0x7ff85c0d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff85bfff640 1 -- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff85c1603a0 msgr2=0x7ff85c157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff860a6e640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff85c1603a0 0x7ff85c157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff85bfff640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff85c1603a0 0x7ff85c157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff85bfff640 1 -- 172.21.15.37:0/1833128601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c0c9180 con 0x7ff85c0cbd60 2024-04-26T20:22:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff85bfff640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff85c0cbd60 0x7ff85c09e080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff844000c00 tx=0x7ff8440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff858ff9640 1 -- 172.21.15.37:0/1833128601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844037040 con 0x7ff85c0cbd60 2024-04-26T20:22:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff858ff9640 1 -- 172.21.15.37:0/1833128601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff844042070 con 0x7ff85c0cbd60 2024-04-26T20:22:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff8624d1640 1 -- 172.21.15.37:0/1833128601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c0c9460 con 0x7ff85c0cbd60 2024-04-26T20:22:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff858ff9640 1 -- 172.21.15.37:0/1833128601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844030880 con 0x7ff85c0cbd60 2024-04-26T20:22:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.077+0000 7ff8624d1640 1 -- 172.21.15.37:0/1833128601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c0c9a20 con 0x7ff85c0cbd60 2024-04-26T20:22:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.078+0000 7ff8624d1640 1 -- 172.21.15.37:0/1833128601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85c06a360 con 0x7ff85c0cbd60 2024-04-26T20:22:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.079+0000 7ff858ff9640 1 -- 172.21.15.37:0/1833128601 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff844030a20 con 0x7ff85c0cbd60 2024-04-26T20:22:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.079+0000 7ff858ff9640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff83c06c920 0x7ff83c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.079+0000 7ff858ff9640 1 -- 172.21.15.37:0/1833128601 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff8440ac390 con 0x7ff85c0cbd60 2024-04-26T20:22:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.079+0000 7ff85b7fe640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff83c06c920 0x7ff83c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.080+0000 7ff85b7fe640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff83c06c920 0x7ff83c06ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0cffe0 tx=0x7ff84c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.083+0000 7ff858ff9640 1 -- 172.21.15.37:0/1833128601 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff84407a7c0 con 0x7ff85c0cbd60 2024-04-26T20:22:23.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.268+0000 7ff8624d1640 1 -- 172.21.15.37:0/1833128601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff85c06e020 con 0x7ff85c0cbd60 2024-04-26T20:22:23.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.268+0000 7ff858ff9640 1 -- 172.21.15.37:0/1833128601 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7ff84407a0c0 con 0x7ff85c0cbd60 2024-04-26T20:22:23.270 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.270+0000 7ff8624d1640 1 -- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff83c06c920 msgr2=0x7ff83c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.270+0000 7ff8624d1640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff83c06c920 0x7ff83c06ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0cffe0 tx=0x7ff84c008040 comp rx=0 tx=0).stop 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.270+0000 7ff8624d1640 1 -- 172.21.15.37:0/1833128601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff85c0cbd60 msgr2=0x7ff85c09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.270+0000 7ff8624d1640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff85c0cbd60 0x7ff85c09e080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff844000c00 tx=0x7ff8440046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.270+0000 7ff8624d1640 1 -- 172.21.15.37:0/1833128601 shutdown_connections 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.270+0000 7ff8624d1640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff85c1603a0 0x7ff85c157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.270+0000 7ff8624d1640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff83c06c920 0x7ff83c06ede0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.270+0000 7ff8624d1640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff85c15ffc0 0x7ff85c0d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.270+0000 7ff8624d1640 1 --2- 172.21.15.37:0/1833128601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff85c0cbd60 0x7ff85c09e080 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.270+0000 7ff8624d1640 1 -- 172.21.15.37:0/1833128601 >> 172.21.15.37:0/1833128601 conn(0x7ff85c0d0d70 msgr2=0x7ff85c0ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.270+0000 7ff8624d1640 1 -- 172.21.15.37:0/1833128601 shutdown_connections 2024-04-26T20:22:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.271+0000 7ff8624d1640 1 -- 172.21.15.37:0/1833128601 wait complete. 2024-04-26T20:22:23.286 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:23.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.391+0000 7fa5a863f640 1 Processor -- start 2024-04-26T20:22:23.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.391+0000 7fa5a863f640 1 -- start start 2024-04-26T20:22:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a863f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa5a00d0f00 0x7fa5a00d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a863f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa5a00d1fe0 0x7fa5a00b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a863f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a00d2ed0 con 0x7fa5a015f470 2024-04-26T20:22:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a863f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a00d31b0 con 0x7fa5a00d0f00 2024-04-26T20:22:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a863f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a00d3490 con 0x7fa5a00d1fe0 2024-04-26T20:22:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a5bda640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa5a00d0f00 0x7fa5a00d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a5bda640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa5a00d0f00 0x7fa5a00d1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60524/0 (socket says 172.21.15.37:60524) 2024-04-26T20:22:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a5bda640 1 -- 172.21.15.37:0/1541648063 learned_addr learned my addr 172.21.15.37:0/1541648063 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a6bdc640 1 --2- 172.21.15.37:0/1541648063 >> v2:172.21.15.134:3300/0 conn(0x7fa5a00d1fe0 0x7fa5a00b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a5bda640 1 -- 172.21.15.37:0/1541648063 >> v2:172.21.15.134:3300/0 conn(0x7fa5a00d1fe0 msgr2=0x7fa5a00b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a5bda640 1 --2- 172.21.15.37:0/1541648063 >> v2:172.21.15.134:3300/0 conn(0x7fa5a00d1fe0 0x7fa5a00b7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a5bda640 1 -- 172.21.15.37:0/1541648063 >> v1:172.21.15.134:6789/0 conn(0x7fa5a015f470 legacy=0x7fa5a00d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.392+0000 7fa5a5bda640 1 -- 172.21.15.37:0/1541648063 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a00d3770 con 0x7fa5a00d0f00 2024-04-26T20:22:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.393+0000 7fa5a5bda640 1 --2- 172.21.15.37:0/1541648063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a00d0f00 0x7fa5a00d1aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0164070 tx=0x7fa59402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa5d2950eb281a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.393+0000 7fa5a53d9640 1 -- 172.21.15.37:0/1541648063 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594037070 con 0x7fa5a00d0f00 2024-04-26T20:22:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.393+0000 7fa5a53d9640 1 -- 172.21.15.37:0/1541648063 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa594032440 con 0x7fa5a00d0f00 2024-04-26T20:22:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.393+0000 7fa5a53d9640 1 -- 172.21.15.37:0/1541648063 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594032810 con 0x7fa5a00d0f00 2024-04-26T20:22:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.394+0000 7fa5a863f640 1 -- 172.21.15.37:0/1541648063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a00d0f00 msgr2=0x7fa5a00d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.394+0000 7fa5a863f640 1 --2- 172.21.15.37:0/1541648063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a00d0f00 0x7fa5a00d1aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0164070 tx=0x7fa59402a230 comp rx=0 tx=0).stop 2024-04-26T20:22:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.394+0000 7fa5a863f640 1 -- 172.21.15.37:0/1541648063 shutdown_connections 2024-04-26T20:22:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.394+0000 7fa5a863f640 1 --2- 172.21.15.37:0/1541648063 >> v2:172.21.15.134:3300/0 conn(0x7fa5a00d1fe0 0x7fa5a00b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.394+0000 7fa5a863f640 1 --2- 172.21.15.37:0/1541648063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a00d0f00 0x7fa5a00d1aa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.394+0000 7fa5a863f640 1 -- 172.21.15.37:0/1541648063 >> 172.21.15.37:0/1541648063 conn(0x7fa5a009e510 msgr2=0x7fa5a00d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.394+0000 7fa5a863f640 1 -- 172.21.15.37:0/1541648063 shutdown_connections 2024-04-26T20:22:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.394+0000 7fa5a863f640 1 -- 172.21.15.37:0/1541648063 wait complete. 2024-04-26T20:22:23.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.395+0000 7fa5a863f640 1 Processor -- start 2024-04-26T20:22:23.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.395+0000 7fa5a863f640 1 -- start start 2024-04-26T20:22:23.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a863f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5a00d0f00 0x7fa5a00c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a863f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5a00d1fe0 0x7fa5a015c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a863f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a015f470 0x7fa5a015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a863f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a00d3f70 con 0x7fa5a015f470 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a863f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a015f850 con 0x7fa5a00d0f00 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a863f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a0158990 con 0x7fa5a00d1fe0 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a6bdc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a015f470 0x7fa5a015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a6bdc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a015f470 0x7fa5a015c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60538/0 (socket says 172.21.15.37:60538) 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a6bdc640 1 -- 172.21.15.37:0/1666424494 learned_addr learned my addr 172.21.15.37:0/1666424494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a5bda640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5a00d1fe0 0x7fa5a015c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a63db640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5a00d0f00 0x7fa5a00c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a6bdc640 1 -- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5a00d1fe0 msgr2=0x7fa5a015c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a6bdc640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5a00d1fe0 0x7fa5a015c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a6bdc640 1 -- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5a00d0f00 msgr2=0x7fa5a00c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a6bdc640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5a00d0f00 0x7fa5a00c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a6bdc640 1 -- 172.21.15.37:0/1666424494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a015cf20 con 0x7fa5a015f470 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.396+0000 7fa5a63db640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5a00d0f00 0x7fa5a00c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.397+0000 7fa5a6bdc640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a015f470 0x7fa5a015c8a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa590007e40 tx=0x7fa59000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.397+0000 7fa59effd640 1 -- 172.21.15.37:0/1666424494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590014620 con 0x7fa5a015f470 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.397+0000 7fa59effd640 1 -- 172.21.15.37:0/1666424494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590019070 con 0x7fa5a015f470 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.397+0000 7fa5a863f640 1 -- 172.21.15.37:0/1666424494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a015d200 con 0x7fa5a015f470 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.397+0000 7fa59effd640 1 -- 172.21.15.37:0/1666424494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590014a20 con 0x7fa5a015f470 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.397+0000 7fa5a863f640 1 -- 172.21.15.37:0/1666424494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a015daa0 con 0x7fa5a015f470 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.398+0000 7fa5a863f640 1 -- 172.21.15.37:0/1666424494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a015d5f0 con 0x7fa5a015f470 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.398+0000 7fa59effd640 1 -- 172.21.15.37:0/1666424494 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa590014bc0 con 0x7fa5a015f470 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.399+0000 7fa59effd640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa58c06c8f0 0x7fa58c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.399+0000 7fa59effd640 1 -- 172.21.15.37:0/1666424494 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa59008c4b0 con 0x7fa5a015f470 2024-04-26T20:22:23.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.399+0000 7fa5a63db640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa58c06c8f0 0x7fa58c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.399+0000 7fa5a63db640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa58c06c8f0 0x7fa58c06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa598003eb0 tx=0x7fa598009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:23.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.403+0000 7fa59effd640 1 -- 172.21.15.37:0/1666424494 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa59005a8e0 con 0x7fa5a015f470 2024-04-26T20:22:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.587+0000 7fa5a863f640 1 -- 172.21.15.37:0/1666424494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa5a0069af0 con 0x7fa5a015f470 2024-04-26T20:22:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.587+0000 7fa59effd640 1 -- 172.21.15.37:0/1666424494 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa5a0069af0 con 0x7fa5a015f470 2024-04-26T20:22:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 -- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa58c06c8f0 msgr2=0x7fa58c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa58c06c8f0 0x7fa58c06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa598003eb0 tx=0x7fa598009040 comp rx=0 tx=0).stop 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 -- 172.21.15.37:0/1666424494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a015f470 msgr2=0x7fa5a015c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a015f470 0x7fa5a015c8a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa590007e40 tx=0x7fa59000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 -- 172.21.15.37:0/1666424494 shutdown_connections 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a015f470 0x7fa5a015c8a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa58c06c8f0 0x7fa58c06edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5a00d1fe0 0x7fa5a015c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 --2- 172.21.15.37:0/1666424494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5a00d0f00 0x7fa5a00c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 -- 172.21.15.37:0/1666424494 >> 172.21.15.37:0/1666424494 conn(0x7fa5a009e510 msgr2=0x7fa5a00d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 -- 172.21.15.37:0/1666424494 shutdown_connections 2024-04-26T20:22:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.589+0000 7fa5a863f640 1 -- 172.21.15.37:0/1666424494 wait complete. 2024-04-26T20:22:23.601 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.710+0000 7f8c1bb4d640 1 Processor -- start 2024-04-26T20:22:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.710+0000 7f8c1bb4d640 1 -- start start 2024-04-26T20:22:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c1bb4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c1415ffc0 0x7f8c140cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c1bb4d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8c140ce940 0x7f8c140cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c1bb4d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c1406f690 con 0x7f8c141603a0 2024-04-26T20:22:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c1bb4d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f8c1406f970 con 0x7f8c140ce940 2024-04-26T20:22:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c1bb4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8c140735a0 con 0x7f8c1415ffc0 2024-04-26T20:22:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c1a0ea640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8c140ce940 0x7f8c140cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c190e8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8c141603a0 0x7f8c14160820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32924/0 (socket says 172.21.15.37:32924) 2024-04-26T20:22:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c190e8640 1 -- 172.21.15.37:0/3337048412 learned_addr learned my addr 172.21.15.37:0/3337048412 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c198e9640 1 --2- 172.21.15.37:0/3337048412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c1415ffc0 0x7f8c140cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c1a0ea640 1 -- 172.21.15.37:0/3337048412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c1415ffc0 msgr2=0x7f8c140cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c1a0ea640 1 --2- 172.21.15.37:0/3337048412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c1415ffc0 0x7f8c140cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c1a0ea640 1 -- 172.21.15.37:0/3337048412 >> v1:172.21.15.37:6789/0 conn(0x7f8c141603a0 legacy=0x7f8c14160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c1a0ea640 1 -- 172.21.15.37:0/3337048412 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c14073880 con 0x7f8c140ce940 2024-04-26T20:22:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.711+0000 7f8c188e7640 1 -- 172.21.15.37:0/3337048412 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1516697451 0 0) 0x7f8c1406f690 con 0x7f8c141603a0 2024-04-26T20:22:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.712+0000 7f8c1a0ea640 1 --2- 172.21.15.37:0/3337048412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c140ce940 0x7f8c140cedb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0097a0 tx=0x7f8c0c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe632d5ec8e69189 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.712+0000 7f8c188e7640 1 -- 172.21.15.37:0/3337048412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c037070 con 0x7f8c140ce940 2024-04-26T20:22:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.712+0000 7f8c188e7640 1 -- 172.21.15.37:0/3337048412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0c032440 con 0x7f8c140ce940 2024-04-26T20:22:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.712+0000 7f8c188e7640 1 -- 172.21.15.37:0/3337048412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c032810 con 0x7f8c140ce940 2024-04-26T20:22:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.712+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/3337048412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c140ce940 msgr2=0x7f8c140cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.712+0000 7f8c1bb4d640 1 --2- 172.21.15.37:0/3337048412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c140ce940 0x7f8c140cedb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0097a0 tx=0x7f8c0c02a180 comp rx=0 tx=0).stop 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.713+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/3337048412 shutdown_connections 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.713+0000 7f8c1bb4d640 1 --2- 172.21.15.37:0/3337048412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c140ce940 0x7f8c140cedb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.713+0000 7f8c1bb4d640 1 --2- 172.21.15.37:0/3337048412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c1415ffc0 0x7f8c140cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.713+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/3337048412 >> 172.21.15.37:0/3337048412 conn(0x7f8c140d0d70 msgr2=0x7f8c140d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.713+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/3337048412 shutdown_connections 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.713+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/3337048412 wait complete. 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.714+0000 7f8c1bb4d640 1 Processor -- start 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.714+0000 7f8c1bb4d640 1 -- start start 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.714+0000 7f8c1bb4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c140ce940 0x7f8c141f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.714+0000 7f8c1bb4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c1415ffc0 0x7f8c1415e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1bb4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c141603a0 0x7f8c140cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1bb4d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1415c7a0 con 0x7f8c141603a0 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1bb4d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1406fc50 con 0x7f8c1415ffc0 2024-04-26T20:22:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1bb4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1415c920 con 0x7f8c140ce940 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1a0ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c141603a0 0x7f8c140cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1a0ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c141603a0 0x7f8c140cfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60556/0 (socket says 172.21.15.37:60556) 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1a0ea640 1 -- 172.21.15.37:0/468206071 learned_addr learned my addr 172.21.15.37:0/468206071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1a0ea640 1 -- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c140ce940 msgr2=0x7f8c141f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1a0ea640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c140ce940 0x7f8c141f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1a0ea640 1 -- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c1415ffc0 msgr2=0x7f8c1415e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1a0ea640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c1415ffc0 0x7f8c1415e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1a0ea640 1 -- 172.21.15.37:0/468206071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1415f350 con 0x7f8c141603a0 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1a0ea640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c141603a0 0x7f8c140cfe20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c035040 tx=0x7f8c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c127fc640 1 -- 172.21.15.37:0/468206071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c037040 con 0x7f8c141603a0 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c127fc640 1 -- 172.21.15.37:0/468206071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0c042070 con 0x7f8c141603a0 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c127fc640 1 -- 172.21.15.37:0/468206071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0308a0 con 0x7f8c141603a0 2024-04-26T20:22:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/468206071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c141f66a0 con 0x7f8c141603a0 2024-04-26T20:22:23.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.715+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/468206071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c141f6c20 con 0x7f8c141603a0 2024-04-26T20:22:23.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.716+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/468206071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1406a360 con 0x7f8c141603a0 2024-04-26T20:22:23.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.717+0000 7f8c127fc640 1 -- 172.21.15.37:0/468206071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c0c030a40 con 0x7f8c141603a0 2024-04-26T20:22:23.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.717+0000 7f8c127fc640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8c0006c920 0x7f8c0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:23.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.717+0000 7f8c127fc640 1 -- 172.21.15.37:0/468206071 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8c0c0ac420 con 0x7f8c141603a0 2024-04-26T20:22:23.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.718+0000 7f8c198e9640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8c0006c920 0x7f8c0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:23.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.718+0000 7f8c198e9640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8c0006c920 0x7f8c0006ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8c04006a90 tx=0x7f8c04006a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.722+0000 7f8c127fc640 1 -- 172.21.15.37:0/468206071 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8c0c07a8e0 con 0x7f8c141603a0 2024-04-26T20:22:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.908+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/468206071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8c140b6cf0 con 0x7f8c141603a0 2024-04-26T20:22:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.908+0000 7f8c127fc640 1 -- 172.21.15.37:0/468206071 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8c0c07a1e0 con 0x7f8c141603a0 2024-04-26T20:22:23.910 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8c0006c920 msgr2=0x7f8c0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8c0006c920 0x7f8c0006ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8c04006a90 tx=0x7f8c04006a20 comp rx=0 tx=0).stop 2024-04-26T20:22:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/468206071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c141603a0 msgr2=0x7f8c140cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c141603a0 0x7f8c140cfe20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c035040 tx=0x7f8c0c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/468206071 shutdown_connections 2024-04-26T20:22:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c141603a0 0x7f8c140cfe20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c1415ffc0 0x7f8c1415e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8c0006c920 0x7f8c0006ede0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 --2- 172.21.15.37:0/468206071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c140ce940 0x7f8c141f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/468206071 >> 172.21.15.37:0/468206071 conn(0x7f8c140d0d70 msgr2=0x7f8c14158330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/468206071 shutdown_connections 2024-04-26T20:22:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:23.911+0000 7f8c1bb4d640 1 -- 172.21.15.37:0/468206071 wait complete. 2024-04-26T20:22:23.923 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:22:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.026+0000 7f0b7d66d640 1 Processor -- start 2024-04-26T20:22:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.026+0000 7f0b7d66d640 1 -- start start 2024-04-26T20:22:24.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b7d66d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0b7806f790 0x7f0b78074030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b7d66d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0b780b6b10 0x7f0b78161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b7d66d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b78161b80 con 0x7f0b78160d70 2024-04-26T20:22:24.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b7d66d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f0b780b4400 con 0x7f0b780b6b10 2024-04-26T20:22:24.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b7d66d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f0b78074570 con 0x7f0b7806f790 2024-04-26T20:22:24.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b777fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0b780b6b10 0x7f0b78161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b777fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0b780b6b10 0x7f0b78161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60572/0 (socket says 172.21.15.37:60572) 2024-04-26T20:22:24.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b777fe640 1 -- 172.21.15.37:0/2303754769 learned_addr learned my addr 172.21.15.37:0/2303754769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:24.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b76ffd640 1 --2- 172.21.15.37:0/2303754769 >> v2:172.21.15.134:3300/0 conn(0x7f0b7806f790 0x7f0b78074030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b777fe640 1 -- 172.21.15.37:0/2303754769 >> v2:172.21.15.134:3300/0 conn(0x7f0b7806f790 msgr2=0x7f0b78074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:24.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b777fe640 1 --2- 172.21.15.37:0/2303754769 >> v2:172.21.15.134:3300/0 conn(0x7f0b7806f790 0x7f0b78074030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b777fe640 1 -- 172.21.15.37:0/2303754769 >> v1:172.21.15.134:6789/0 conn(0x7f0b78160d70 legacy=0x7f0b7815d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.027+0000 7f0b777fe640 1 -- 172.21.15.37:0/2303754769 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b780b4120 con 0x7f0b780b6b10 2024-04-26T20:22:24.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.028+0000 7f0b777fe640 1 --2- 172.21.15.37:0/2303754769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780b6b10 0x7f0b78161200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0b78161d50 tx=0x7f0b7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1552d8df8f1ea33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:24.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.028+0000 7f0b75ffb640 1 -- 172.21.15.37:0/2303754769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70037070 con 0x7f0b780b6b10 2024-04-26T20:22:24.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.028+0000 7f0b75ffb640 1 -- 172.21.15.37:0/2303754769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b70032440 con 0x7f0b780b6b10 2024-04-26T20:22:24.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.028+0000 7f0b75ffb640 1 -- 172.21.15.37:0/2303754769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70032810 con 0x7f0b780b6b10 2024-04-26T20:22:24.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.029+0000 7f0b7d66d640 1 -- 172.21.15.37:0/2303754769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780b6b10 msgr2=0x7f0b78161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.029+0000 7f0b7d66d640 1 --2- 172.21.15.37:0/2303754769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780b6b10 0x7f0b78161200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0b78161d50 tx=0x7f0b7002a270 comp rx=0 tx=0).stop 2024-04-26T20:22:24.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.029+0000 7f0b7d66d640 1 -- 172.21.15.37:0/2303754769 shutdown_connections 2024-04-26T20:22:24.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.029+0000 7f0b7d66d640 1 --2- 172.21.15.37:0/2303754769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780b6b10 0x7f0b78161200 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.029+0000 7f0b7d66d640 1 --2- 172.21.15.37:0/2303754769 >> v2:172.21.15.134:3300/0 conn(0x7f0b7806f790 0x7f0b78074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.029+0000 7f0b7d66d640 1 -- 172.21.15.37:0/2303754769 >> 172.21.15.37:0/2303754769 conn(0x7f0b780d0d50 msgr2=0x7f0b780d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:24.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.029+0000 7f0b7d66d640 1 -- 172.21.15.37:0/2303754769 shutdown_connections 2024-04-26T20:22:24.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.029+0000 7f0b7d66d640 1 -- 172.21.15.37:0/2303754769 wait complete. 2024-04-26T20:22:24.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.030+0000 7f0b7d66d640 1 Processor -- start 2024-04-26T20:22:24.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.030+0000 7f0b7d66d640 1 -- start start 2024-04-26T20:22:24.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b7d66d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b7806f790 0x7f0b780cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b7d66d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780b6b10 0x7f0b78144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b7d66d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b78160d70 0x7f0b7815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b7d66d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0b780735b0 con 0x7f0b780b6b10 2024-04-26T20:22:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b7d66d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b780732d0 con 0x7f0b7806f790 2024-04-26T20:22:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b7d66d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0b780b3d80 con 0x7f0b78160d70 2024-04-26T20:22:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b767fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780b6b10 0x7f0b78144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b767fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780b6b10 0x7f0b78144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60574/0 (socket says 172.21.15.37:60574) 2024-04-26T20:22:24.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b767fc640 1 -- 172.21.15.37:0/428105363 learned_addr learned my addr 172.21.15.37:0/428105363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:24.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b777fe640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b78160d70 0x7f0b7815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b76ffd640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b7806f790 0x7f0b780cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.031+0000 7f0b767fc640 1 -- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b78160d70 msgr2=0x7f0b7815c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.032+0000 7f0b767fc640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b78160d70 0x7f0b7815c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.032+0000 7f0b767fc640 1 -- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b7806f790 msgr2=0x7f0b780cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.032+0000 7f0b767fc640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b7806f790 0x7f0b780cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.032+0000 7f0b767fc640 1 -- 172.21.15.37:0/428105363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b780acf80 con 0x7f0b780b6b10 2024-04-26T20:22:24.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.032+0000 7f0b767fc640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780b6b10 0x7f0b78144240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c400 tx=0x7f0b6000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:24.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.032+0000 7f0b6f7fe640 1 -- 172.21.15.37:0/428105363 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b600052d0 con 0x7f0b780b6b10 2024-04-26T20:22:24.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.032+0000 7f0b6f7fe640 1 -- 172.21.15.37:0/428105363 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b60005490 con 0x7f0b780b6b10 2024-04-26T20:22:24.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.032+0000 7f0b7d66d640 1 -- 172.21.15.37:0/428105363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b780ce9c0 con 0x7f0b780b6b10 2024-04-26T20:22:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.032+0000 7f0b6f7fe640 1 -- 172.21.15.37:0/428105363 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60007260 con 0x7f0b780b6b10 2024-04-26T20:22:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.032+0000 7f0b7d66d640 1 -- 172.21.15.37:0/428105363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b780cef80 con 0x7f0b780b6b10 2024-04-26T20:22:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.033+0000 7f0b7d66d640 1 -- 172.21.15.37:0/428105363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7806a360 con 0x7f0b780b6b10 2024-04-26T20:22:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.033+0000 7f0b6f7fe640 1 -- 172.21.15.37:0/428105363 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b60007400 con 0x7f0b780b6b10 2024-04-26T20:22:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.034+0000 7f0b6f7fe640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0b5806c8f0 0x7f0b5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.034+0000 7f0b6f7fe640 1 -- 172.21.15.37:0/428105363 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0b6008b050 con 0x7f0b780b6b10 2024-04-26T20:22:24.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.034+0000 7f0b76ffd640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0b5806c8f0 0x7f0b5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.035+0000 7f0b76ffd640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0b5806c8f0 0x7f0b5806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0b680073f0 tx=0x7f0b6800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:24.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.038+0000 7f0b6f7fe640 1 -- 172.21.15.37:0/428105363 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b60059510 con 0x7f0b780b6b10 2024-04-26T20:22:24.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.238+0000 7f0b7d66d640 1 -- 172.21.15.37:0/428105363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0b7806ff40 con 0x7f0b780b6b10 2024-04-26T20:22:24.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.238+0000 7f0b6f7fe640 1 -- 172.21.15.37:0/428105363 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f0b60058e10 con 0x7f0b780b6b10 2024-04-26T20:22:24.240 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 -- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0b5806c8f0 msgr2=0x7f0b5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0b5806c8f0 0x7f0b5806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0b680073f0 tx=0x7f0b6800b040 comp rx=0 tx=0).stop 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 -- 172.21.15.37:0/428105363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780b6b10 msgr2=0x7f0b78144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780b6b10 0x7f0b78144240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c400 tx=0x7f0b6000c8c0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 -- 172.21.15.37:0/428105363 shutdown_connections 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0b5806c8f0 0x7f0b5806edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b78160d70 0x7f0b7815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780b6b10 0x7f0b78144240 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 --2- 172.21.15.37:0/428105363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b7806f790 0x7f0b780cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 -- 172.21.15.37:0/428105363 >> 172.21.15.37:0/428105363 conn(0x7f0b780d0d50 msgr2=0x7f0b780cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 -- 172.21.15.37:0/428105363 shutdown_connections 2024-04-26T20:22:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.240+0000 7f0b7d66d640 1 -- 172.21.15.37:0/428105363 wait complete. 2024-04-26T20:22:24.252 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T20:22:24.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:24.253 DEBUG:teuthology.orchestra.run.smithi134:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-26T20:22:24.283 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:22:24.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:24.283 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T20:22:24.403 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:24.403 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:24.419 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:24.419 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:24.475 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:24.475 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:24.531 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.670+0000 7fc3f8124640 1 Processor -- start 2024-04-26T20:22:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.670+0000 7fc3f8124640 1 -- start start 2024-04-26T20:22:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f8124640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc3f0161200 0x7fc3f00b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f8124640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f0161b80 con 0x7fc3f006f790 2024-04-26T20:22:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f8124640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f00b4400 con 0x7fc3f0160d70 2024-04-26T20:22:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f8124640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f00b4120 con 0x7fc3f0161200 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f5ec0640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc3f006f790 0x7fc3f0074030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32932/0 (socket says 172.21.15.37:32932) 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f5ec0640 1 -- 172.21.15.37:0/926492689 learned_addr learned my addr 172.21.15.37:0/926492689 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1558254084 0 0) 0x7fc3f0161b80 con 0x7fc3f006f790 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3d8003520 con 0x7fc3f006f790 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f66c1640 1 --2- 172.21.15.37:0/926492689 >> v2:172.21.15.134:3300/0 conn(0x7fc3f0161200 0x7fc3f00b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1525133673 0 0) 0x7fc3d8003520 con 0x7fc3f006f790 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 >> v2:172.21.15.134:3300/0 conn(0x7fc3f0161200 msgr2=0x7fc3f00b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 --2- 172.21.15.37:0/926492689 >> v2:172.21.15.134:3300/0 conn(0x7fc3f0161200 0x7fc3f00b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 >> v1:172.21.15.134:6789/0 conn(0x7fc3f0160d70 legacy=0x7fc3f00b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f0072ff0 con 0x7fc3f006f790 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fc3e8001670 con 0x7fc3f006f790 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 >> v1:172.21.15.37:6789/0 conn(0x7fc3f006f790 legacy=0x7fc3f0074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d80039b0 0x7fc3d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3d8004800 0x7fc3d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 0x7fc3d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f00b4120 con 0x7fc3d80051c0 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d800c060 con 0x7fc3d80039b0 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.671+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d800c260 con 0x7fc3d8004800 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.672+0000 7fc3f66c1640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 0x7fc3d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.672+0000 7fc3f5ec0640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d80039b0 0x7fc3d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.672+0000 7fc3f56bf640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3d8004800 0x7fc3d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.672+0000 7fc3f66c1640 1 -- 172.21.15.37:0/926492689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3d8004800 msgr2=0x7fc3d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.672+0000 7fc3f66c1640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3d8004800 0x7fc3d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.672+0000 7fc3f66c1640 1 -- 172.21.15.37:0/926492689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d80039b0 msgr2=0x7fc3d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.672+0000 7fc3f66c1640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d80039b0 0x7fc3d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.672+0000 7fc3f66c1640 1 -- 172.21.15.37:0/926492689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d8003520 con 0x7fc3d80051c0 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.672+0000 7fc3f56bf640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3d8004800 0x7fc3d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.672+0000 7fc3f66c1640 1 -- 172.21.15.37:0/926492689 reap_dead start 2024-04-26T20:22:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.673+0000 7fc3f66c1640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 0x7fc3d800b9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009fc0 tx=0x7fc3d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90b6174502b942d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.673+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0035020 con 0x7fc3d80051c0 2024-04-26T20:22:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.673+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e0003b90 con 0x7fc3d80051c0 2024-04-26T20:22:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.673+0000 7fc3f4ebe640 1 -- 172.21.15.37:0/926492689 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0009c10 con 0x7fc3d80051c0 2024-04-26T20:22:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.673+0000 7fc3f8124640 1 -- 172.21.15.37:0/926492689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 msgr2=0x7fc3d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.673+0000 7fc3f8124640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 0x7fc3d800b9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009fc0 tx=0x7fc3d8005830 comp rx=0 tx=0).stop 2024-04-26T20:22:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.673+0000 7fc3f8124640 1 -- 172.21.15.37:0/926492689 shutdown_connections 2024-04-26T20:22:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.673+0000 7fc3f8124640 1 --2- 172.21.15.37:0/926492689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 0x7fc3d800b9c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.673+0000 7fc3f8124640 1 -- 172.21.15.37:0/926492689 >> 172.21.15.37:0/926492689 conn(0x7fc3f00d0d50 msgr2=0x7fc3f00d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.673+0000 7fc3f8124640 1 -- 172.21.15.37:0/926492689 shutdown_connections 2024-04-26T20:22:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.673+0000 7fc3f8124640 1 -- 172.21.15.37:0/926492689 wait complete. 2024-04-26T20:22:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.674+0000 7fc3f8124640 1 Processor -- start 2024-04-26T20:22:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f8124640 1 -- start start 2024-04-26T20:22:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f8124640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 0x7fc3f013b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f8124640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3f01542f0 0x7fc3f013baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f8124640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f0152890 0x7fc3f014fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f8124640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f0161650 con 0x7fc3d80051c0 2024-04-26T20:22:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f8124640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f00735b0 con 0x7fc3f01542f0 2024-04-26T20:22:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f8124640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f00b3d80 con 0x7fc3f0152890 2024-04-26T20:22:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f5ec0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 0x7fc3f013b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f5ec0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 0x7fc3f013b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60602/0 (socket says 172.21.15.37:60602) 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f5ec0640 1 -- 172.21.15.37:0/2731549969 learned_addr learned my addr 172.21.15.37:0/2731549969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f66c1640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f0152890 0x7fc3f014fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.675+0000 7fc3f56bf640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3f01542f0 0x7fc3f013baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.676+0000 7fc3f5ec0640 1 -- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f0152890 msgr2=0x7fc3f014fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.676+0000 7fc3f5ec0640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f0152890 0x7fc3f014fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.676+0000 7fc3f5ec0640 1 -- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3f01542f0 msgr2=0x7fc3f013baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.676+0000 7fc3f5ec0640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3f01542f0 0x7fc3f013baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.676+0000 7fc3f5ec0640 1 -- 172.21.15.37:0/2731549969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f013b810 con 0x7fc3d80051c0 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.676+0000 7fc3f5ec0640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 0x7fc3f013b2d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80072f0 tx=0x7fc3e80077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.676+0000 7fc3ee7fc640 1 -- 172.21.15.37:0/2731549969 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8001660 con 0x7fc3d80051c0 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.676+0000 7fc3ee7fc640 1 -- 172.21.15.37:0/2731549969 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e8018070 con 0x7fc3d80051c0 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.676+0000 7fc3f8124640 1 -- 172.21.15.37:0/2731549969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f014dca0 con 0x7fc3d80051c0 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.676+0000 7fc3ee7fc640 1 -- 172.21.15.37:0/2731549969 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800fb80 con 0x7fc3d80051c0 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.676+0000 7fc3f8124640 1 -- 172.21.15.37:0/2731549969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f014e260 con 0x7fc3d80051c0 2024-04-26T20:22:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.677+0000 7fc3f8124640 1 -- 172.21.15.37:0/2731549969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f006a360 con 0x7fc3d80051c0 2024-04-26T20:22:24.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.678+0000 7fc3ee7fc640 1 -- 172.21.15.37:0/2731549969 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3e800fd20 con 0x7fc3d80051c0 2024-04-26T20:22:24.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.678+0000 7fc3ee7fc640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc3dc06cac0 0x7fc3dc06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.678+0000 7fc3ee7fc640 1 -- 172.21.15.37:0/2731549969 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc3e808b930 con 0x7fc3d80051c0 2024-04-26T20:22:24.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.678+0000 7fc3f56bf640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc3dc06cac0 0x7fc3dc06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.679+0000 7fc3f56bf640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc3dc06cac0 0x7fc3dc06ef80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc3f01546f0 tx=0x7fc3e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:24.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.682+0000 7fc3ee7fc640 1 -- 172.21.15.37:0/2731549969 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc3f013b810 con 0x7fc3d80051c0 2024-04-26T20:22:24.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.867+0000 7fc3f8124640 1 -- 172.21.15.37:0/2731549969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3f006ff40 con 0x7fc3d80051c0 2024-04-26T20:22:24.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.868+0000 7fc3ee7fc640 1 -- 172.21.15.37:0/2731549969 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7fc3e80597a0 con 0x7fc3d80051c0 2024-04-26T20:22:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:24.870 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.870+0000 7fc3f8124640 1 -- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc3dc06cac0 msgr2=0x7fc3dc06ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.870+0000 7fc3f8124640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc3dc06cac0 0x7fc3dc06ef80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc3f01546f0 tx=0x7fc3e4009040 comp rx=0 tx=0).stop 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.870+0000 7fc3f8124640 1 -- 172.21.15.37:0/2731549969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 msgr2=0x7fc3f013b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.870+0000 7fc3f8124640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 0x7fc3f013b2d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80072f0 tx=0x7fc3e80077b0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.870+0000 7fc3f8124640 1 -- 172.21.15.37:0/2731549969 shutdown_connections 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.870+0000 7fc3f8124640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc3dc06cac0 0x7fc3dc06ef80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.870+0000 7fc3f8124640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f0152890 0x7fc3f014fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.870+0000 7fc3f8124640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3f01542f0 0x7fc3f013baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.870+0000 7fc3f8124640 1 --2- 172.21.15.37:0/2731549969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d80051c0 0x7fc3f013b2d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.870+0000 7fc3f8124640 1 -- 172.21.15.37:0/2731549969 >> 172.21.15.37:0/2731549969 conn(0x7fc3f00d0d50 msgr2=0x7fc3f00d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.871+0000 7fc3f8124640 1 -- 172.21.15.37:0/2731549969 shutdown_connections 2024-04-26T20:22:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.871+0000 7fc3f8124640 1 -- 172.21.15.37:0/2731549969 wait complete. 2024-04-26T20:22:24.883 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:24.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:24.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.986+0000 7f6fd38f2640 1 Processor -- start 2024-04-26T20:22:24.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.987+0000 7f6fd38f2640 1 -- start start 2024-04-26T20:22:24.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.987+0000 7f6fd38f2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6fcc0b2290 0x7f6fcc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.987+0000 7f6fd38f2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fcc0d26f0 0x7f6fcc0cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.987+0000 7f6fd38f2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fcc0cd640 con 0x7f6fcc0cf870 2024-04-26T20:22:24.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.987+0000 7f6fd38f2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6fcc0cd920 con 0x7f6fcc0b2290 2024-04-26T20:22:24.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.987+0000 7f6fd38f2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc0cdc00 con 0x7f6fcc0d26f0 2024-04-26T20:22:24.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.988+0000 7f6fd168e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6fcc0b2290 0x7f6fcc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.988+0000 7f6fd168e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6fcc0b2290 0x7f6fcc0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60604/0 (socket says 172.21.15.37:60604) 2024-04-26T20:22:24.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.988+0000 7f6fd168e640 1 -- 172.21.15.37:0/3579274826 learned_addr learned my addr 172.21.15.37:0/3579274826 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:24.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.988+0000 7f6fd168e640 1 -- 172.21.15.37:0/3579274826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fcc0d26f0 msgr2=0x7f6fcc0cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:24.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.988+0000 7f6fd168e640 1 --2- 172.21.15.37:0/3579274826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fcc0d26f0 0x7f6fcc0cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.988+0000 7f6fd168e640 1 -- 172.21.15.37:0/3579274826 >> v1:172.21.15.134:6789/0 conn(0x7f6fcc0cf870 legacy=0x7f6fcc0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:24.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.988+0000 7f6fd168e640 1 -- 172.21.15.37:0/3579274826 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc0cdee0 con 0x7f6fcc0b2290 2024-04-26T20:22:24.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.988+0000 7f6fd168e640 1 --2- 172.21.15.37:0/3579274826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fcc0b2290 0x7f6fcc0b2930 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009870 tx=0x7f6fc402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84aaf0ffd2f9b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:24.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.989+0000 7f6fcbfff640 1 -- 172.21.15.37:0/3579274826 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4037070 con 0x7f6fcc0b2290 2024-04-26T20:22:24.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.989+0000 7f6fcbfff640 1 -- 172.21.15.37:0/3579274826 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc4032440 con 0x7f6fcc0b2290 2024-04-26T20:22:24.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.989+0000 7f6fcbfff640 1 -- 172.21.15.37:0/3579274826 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4032810 con 0x7f6fcc0b2290 2024-04-26T20:22:24.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.989+0000 7f6fd38f2640 1 -- 172.21.15.37:0/3579274826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fcc0b2290 msgr2=0x7f6fcc0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.989+0000 7f6fd38f2640 1 --2- 172.21.15.37:0/3579274826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fcc0b2290 0x7f6fcc0b2930 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009870 tx=0x7f6fc402a210 comp rx=0 tx=0).stop 2024-04-26T20:22:24.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.989+0000 7f6fd38f2640 1 -- 172.21.15.37:0/3579274826 shutdown_connections 2024-04-26T20:22:24.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.989+0000 7f6fd38f2640 1 --2- 172.21.15.37:0/3579274826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fcc0d26f0 0x7f6fcc0cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.989+0000 7f6fd38f2640 1 --2- 172.21.15.37:0/3579274826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fcc0b2290 0x7f6fcc0b2930 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.989+0000 7f6fd38f2640 1 -- 172.21.15.37:0/3579274826 >> 172.21.15.37:0/3579274826 conn(0x7f6fcc0d2ff0 msgr2=0x7f6fcc0d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:24.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.989+0000 7f6fd38f2640 1 -- 172.21.15.37:0/3579274826 shutdown_connections 2024-04-26T20:22:24.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.989+0000 7f6fd38f2640 1 -- 172.21.15.37:0/3579274826 wait complete. 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.990+0000 7f6fd38f2640 1 Processor -- start 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd38f2640 1 -- start start 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd38f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fcc0b2290 0x7f6fcc1f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd38f2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fcc0cf870 0x7f6fcc15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd38f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fcc0d26f0 0x7f6fcc0cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd38f2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc15ec60 con 0x7f6fcc0d26f0 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd38f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc0b2670 con 0x7f6fcc0b2290 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd38f2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc15ede0 con 0x7f6fcc0cf870 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd1e8f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fcc0d26f0 0x7f6fcc0cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd1e8f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fcc0d26f0 0x7f6fcc0cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60612/0 (socket says 172.21.15.37:60612) 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd1e8f640 1 -- 172.21.15.37:0/170636722 learned_addr learned my addr 172.21.15.37:0/170636722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd168e640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fcc0b2290 0x7f6fcc1f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.991+0000 7f6fd0e8d640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fcc0cf870 0x7f6fcc15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.992+0000 7f6fd1e8f640 1 -- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fcc0cf870 msgr2=0x7f6fcc15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.992+0000 7f6fd1e8f640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fcc0cf870 0x7f6fcc15d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.992+0000 7f6fd1e8f640 1 -- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fcc0b2290 msgr2=0x7f6fcc1f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.992+0000 7f6fd1e8f640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fcc0b2290 0x7f6fcc1f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.992+0000 7f6fd1e8f640 1 -- 172.21.15.37:0/170636722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc0cb620 con 0x7f6fcc0d26f0 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.992+0000 7f6fd1e8f640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fcc0d26f0 0x7f6fcc0cabd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00b970 tx=0x7f6fbc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.992+0000 7f6fc9ffb640 1 -- 172.21.15.37:0/170636722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc019070 con 0x7f6fcc0d26f0 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.992+0000 7f6fc9ffb640 1 -- 172.21.15.37:0/170636722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fbc00cb60 con 0x7f6fcc0d26f0 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.992+0000 7f6fd38f2640 1 -- 172.21.15.37:0/170636722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fcc0cb900 con 0x7f6fcc0d26f0 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.992+0000 7f6fc9ffb640 1 -- 172.21.15.37:0/170636722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc014440 con 0x7f6fcc0d26f0 2024-04-26T20:22:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.992+0000 7f6fd38f2640 1 -- 172.21.15.37:0/170636722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fcc0b7b40 con 0x7f6fcc0d26f0 2024-04-26T20:22:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.993+0000 7f6fd38f2640 1 -- 172.21.15.37:0/170636722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fcc0b7d40 con 0x7f6fcc0d26f0 2024-04-26T20:22:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.994+0000 7f6fc9ffb640 1 -- 172.21.15.37:0/170636722 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fbc01c070 con 0x7f6fcc0d26f0 2024-04-26T20:22:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.994+0000 7f6fc9ffb640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6fb806c8f0 0x7f6fb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.994+0000 7f6fc9ffb640 1 -- 172.21.15.37:0/170636722 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6fbc08b810 con 0x7f6fcc0d26f0 2024-04-26T20:22:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.994+0000 7f6fd168e640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6fb806c8f0 0x7f6fb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.995+0000 7f6fd168e640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6fb806c8f0 0x7f6fb806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4034000 tx=0x7f6fc402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:24.999+0000 7f6fc9ffb640 1 -- 172.21.15.37:0/170636722 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6fbc0595f0 con 0x7f6fcc0d26f0 2024-04-26T20:22:25.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.183+0000 7f6fd38f2640 1 -- 172.21.15.37:0/170636722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6fcc06a360 con 0x7f6fcc0d26f0 2024-04-26T20:22:25.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.183+0000 7f6fc9ffb640 1 -- 172.21.15.37:0/170636722 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f6fbc05d8f0 con 0x7f6fcc0d26f0 2024-04-26T20:22:25.185 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:25.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 -- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6fb806c8f0 msgr2=0x7f6fb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6fb806c8f0 0x7f6fb806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4034000 tx=0x7f6fc402e000 comp rx=0 tx=0).stop 2024-04-26T20:22:25.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 -- 172.21.15.37:0/170636722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fcc0d26f0 msgr2=0x7f6fcc0cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fcc0d26f0 0x7f6fcc0cabd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00b970 tx=0x7f6fbc00be30 comp rx=0 tx=0).stop 2024-04-26T20:22:25.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 -- 172.21.15.37:0/170636722 shutdown_connections 2024-04-26T20:22:25.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fcc0d26f0 0x7f6fcc0cabd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6fb806c8f0 0x7f6fb806edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fcc0cf870 0x7f6fcc15d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 --2- 172.21.15.37:0/170636722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fcc0b2290 0x7f6fcc1f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 -- 172.21.15.37:0/170636722 >> 172.21.15.37:0/170636722 conn(0x7f6fcc0d2ff0 msgr2=0x7f6fcc0afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:25.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 -- 172.21.15.37:0/170636722 shutdown_connections 2024-04-26T20:22:25.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.185+0000 7f6fd38f2640 1 -- 172.21.15.37:0/170636722 wait complete. 2024-04-26T20:22:25.197 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:25.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.300+0000 7faa05c59640 1 Processor -- start 2024-04-26T20:22:25.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.300+0000 7faa05c59640 1 -- start start 2024-04-26T20:22:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7faa05c59640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa0015f470 0x7faa000d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7faa05c59640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faa000d0d30 0x7faa000d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7faa05c59640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa000d2ed0 con 0x7faa000d1fe0 2024-04-26T20:22:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7faa05c59640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7faa000d1190 con 0x7faa000d0d30 2024-04-26T20:22:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7faa05c59640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faa000d31b0 con 0x7faa0015f470 2024-04-26T20:22:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7fa9feffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faa000d0d30 0x7faa000d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7fa9fffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7faa000d1fe0 0x7faa00162fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32946/0 (socket says 172.21.15.37:32946) 2024-04-26T20:22:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7fa9fffff640 1 -- 172.21.15.37:0/727952536 learned_addr learned my addr 172.21.15.37:0/727952536 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7fa9ff7fe640 1 --2- 172.21.15.37:0/727952536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa0015f470 0x7faa000d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7fa9feffd640 1 -- 172.21.15.37:0/727952536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa0015f470 msgr2=0x7faa000d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7fa9feffd640 1 --2- 172.21.15.37:0/727952536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa0015f470 0x7faa000d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7fa9feffd640 1 -- 172.21.15.37:0/727952536 >> v1:172.21.15.37:6789/0 conn(0x7faa000d1fe0 legacy=0x7faa00162fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7fa9fe7fc640 1 -- 172.21.15.37:0/727952536 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3946835901 0 0) 0x7faa000d2ed0 con 0x7faa000d1fe0 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.301+0000 7fa9feffd640 1 -- 172.21.15.37:0/727952536 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa000d3490 con 0x7faa000d0d30 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.302+0000 7fa9feffd640 1 --2- 172.21.15.37:0/727952536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa000d0d30 0x7faa000d1aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40097d0 tx=0x7fa9f402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbce84ea37ba1ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.302+0000 7fa9fe7fc640 1 -- 172.21.15.37:0/727952536 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4037070 con 0x7faa000d0d30 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.302+0000 7fa9fe7fc640 1 -- 172.21.15.37:0/727952536 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f4032440 con 0x7faa000d0d30 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.302+0000 7fa9fe7fc640 1 -- 172.21.15.37:0/727952536 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4032810 con 0x7faa000d0d30 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.303+0000 7faa05c59640 1 -- 172.21.15.37:0/727952536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa000d0d30 msgr2=0x7faa000d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.303+0000 7faa05c59640 1 --2- 172.21.15.37:0/727952536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa000d0d30 0x7faa000d1aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40097d0 tx=0x7fa9f402a1c0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.303+0000 7faa05c59640 1 -- 172.21.15.37:0/727952536 shutdown_connections 2024-04-26T20:22:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.303+0000 7faa05c59640 1 --2- 172.21.15.37:0/727952536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa000d0d30 0x7faa000d1aa0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.303+0000 7faa05c59640 1 --2- 172.21.15.37:0/727952536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa0015f470 0x7faa000d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.303+0000 7faa05c59640 1 -- 172.21.15.37:0/727952536 >> 172.21.15.37:0/727952536 conn(0x7faa0009e510 msgr2=0x7faa000d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.303+0000 7faa05c59640 1 -- 172.21.15.37:0/727952536 shutdown_connections 2024-04-26T20:22:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.303+0000 7faa05c59640 1 -- 172.21.15.37:0/727952536 wait complete. 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.304+0000 7faa05c59640 1 Processor -- start 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.304+0000 7faa05c59640 1 -- start start 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7faa05c59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa000d0d30 0x7faa00156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7faa05c59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa000d1fe0 0x7faa000ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7faa05c59640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa0015f470 0x7faa000ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7faa05c59640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faa0015f850 con 0x7faa000d1fe0 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7faa05c59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa000d3a50 con 0x7faa000d0d30 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7faa05c59640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faa000d3f70 con 0x7faa0015f470 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7fa9feffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa000d1fe0 0x7faa000ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7fa9feffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa000d1fe0 0x7faa000ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60626/0 (socket says 172.21.15.37:60626) 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7fa9feffd640 1 -- 172.21.15.37:0/4285863864 learned_addr learned my addr 172.21.15.37:0/4285863864 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7fa9feffd640 1 -- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa0015f470 msgr2=0x7faa000ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7fa9feffd640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa0015f470 0x7faa000ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7fa9feffd640 1 -- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa000d0d30 msgr2=0x7faa00156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7fa9ff7fe640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa000d0d30 0x7faa00156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7fa9feffd640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa000d0d30 0x7faa00156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7fa9feffd640 1 -- 172.21.15.37:0/4285863864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa00157010 con 0x7faa000d1fe0 2024-04-26T20:22:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7fa9ff7fe640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa000d0d30 0x7faa00156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.305+0000 7fa9feffd640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa000d1fe0 0x7faa000ce3c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40096f0 tx=0x7fa9f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.306+0000 7fa9e7fff640 1 -- 172.21.15.37:0/4285863864 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4037040 con 0x7faa000d1fe0 2024-04-26T20:22:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.306+0000 7fa9e7fff640 1 -- 172.21.15.37:0/4285863864 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f4030500 con 0x7faa000d1fe0 2024-04-26T20:22:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.306+0000 7faa05c59640 1 -- 172.21.15.37:0/4285863864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa001f5290 con 0x7faa000d1fe0 2024-04-26T20:22:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.306+0000 7fa9e7fff640 1 -- 172.21.15.37:0/4285863864 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40308d0 con 0x7faa000d1fe0 2024-04-26T20:22:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.306+0000 7faa05c59640 1 -- 172.21.15.37:0/4285863864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001f5850 con 0x7faa000d1fe0 2024-04-26T20:22:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.307+0000 7faa05c59640 1 -- 172.21.15.37:0/4285863864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0006a360 con 0x7faa000d1fe0 2024-04-26T20:22:25.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.307+0000 7fa9e7fff640 1 -- 172.21.15.37:0/4285863864 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9f4044020 con 0x7faa000d1fe0 2024-04-26T20:22:25.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.308+0000 7fa9e7fff640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9ec06c8f0 0x7fa9ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.308+0000 7fa9e7fff640 1 -- 172.21.15.37:0/4285863864 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9f40abb30 con 0x7faa000d1fe0 2024-04-26T20:22:25.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.308+0000 7fa9ff7fe640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9ec06c8f0 0x7fa9ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.308+0000 7fa9ff7fe640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9ec06c8f0 0x7fa9ec06edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009a20 tx=0x7fa9f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:25.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.312+0000 7fa9e7fff640 1 -- 172.21.15.37:0/4285863864 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9f404b020 con 0x7faa000d1fe0 2024-04-26T20:22:25.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.495+0000 7faa05c59640 1 -- 172.21.15.37:0/4285863864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faa0006dbc0 con 0x7faa000d1fe0 2024-04-26T20:22:25.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.495+0000 7fa9e7fff640 1 -- 172.21.15.37:0/4285863864 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7faa00157010 con 0x7faa000d1fe0 2024-04-26T20:22:25.497 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:25.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 -- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9ec06c8f0 msgr2=0x7fa9ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9ec06c8f0 0x7fa9ec06edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009a20 tx=0x7fa9f0008040 comp rx=0 tx=0).stop 2024-04-26T20:22:25.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 -- 172.21.15.37:0/4285863864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa000d1fe0 msgr2=0x7faa000ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa000d1fe0 0x7faa000ce3c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40096f0 tx=0x7fa9f40046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 -- 172.21.15.37:0/4285863864 shutdown_connections 2024-04-26T20:22:25.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9ec06c8f0 0x7fa9ec06edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa0015f470 0x7faa000ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa000d1fe0 0x7faa000ce3c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 --2- 172.21.15.37:0/4285863864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa000d0d30 0x7faa00156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 -- 172.21.15.37:0/4285863864 >> 172.21.15.37:0/4285863864 conn(0x7faa0009e510 msgr2=0x7faa000d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:25.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 -- 172.21.15.37:0/4285863864 shutdown_connections 2024-04-26T20:22:25.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.498+0000 7faa05c59640 1 -- 172.21.15.37:0/4285863864 wait complete. 2024-04-26T20:22:25.510 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:25.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.612+0000 7fd8f75b7640 1 Processor -- start 2024-04-26T20:22:25.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.612+0000 7fd8f75b7640 1 -- start start 2024-04-26T20:22:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.613+0000 7fd8f75b7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f01610c0 0x7fd8f00d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.613+0000 7fd8f75b7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f0161510 con 0x7fd8f0160800 2024-04-26T20:22:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.613+0000 7fd8f75b7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f00d0930 con 0x7fd8f00d2150 2024-04-26T20:22:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.613+0000 7fd8f75b7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f00d2590 con 0x7fd8f01610c0 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.613+0000 7fd8f5353640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd8f0160800 0x7fd8f00d5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32952/0 (socket says 172.21.15.37:32952) 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.613+0000 7fd8f5353640 1 -- 172.21.15.37:0/3216768757 learned_addr learned my addr 172.21.15.37:0/3216768757 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8f4b52640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f01610c0 0x7fd8f00d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 676176982 0 0) 0x7fd8f0161510 con 0x7fd8f0160800 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8d8003520 con 0x7fd8f0160800 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 365458125 0 0) 0x7fd8d8003520 con 0x7fd8f0160800 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f01610c0 msgr2=0x7fd8f00d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f01610c0 0x7fd8f00d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 >> v1:172.21.15.134:6789/0 conn(0x7fd8f00d2150 legacy=0x7fd8f00d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f00d0c00 con 0x7fd8f0160800 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fd8e8001670 con 0x7fd8f0160800 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 >> v1:172.21.15.37:6789/0 conn(0x7fd8f0160800 legacy=0x7fd8f00d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f01610c0 0x7fd8d80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8d8003f60 0x7fd8d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8d8004d40 0x7fd8d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f00d2590 con 0x7fd8d8003f60 2024-04-26T20:22:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d800baa0 con 0x7fd8d8004d40 2024-04-26T20:22:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d800bca0 con 0x7fd8f01610c0 2024-04-26T20:22:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8f4b52640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8d8003f60 0x7fd8d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8f5353640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f01610c0 0x7fd8d80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8f5b54640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8d8004d40 0x7fd8d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8f4b52640 1 -- 172.21.15.37:0/3216768757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f01610c0 msgr2=0x7fd8d80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8f4b52640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f01610c0 0x7fd8d80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8f4b52640 1 -- 172.21.15.37:0/3216768757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8d8004d40 msgr2=0x7fd8d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8f4b52640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8d8004d40 0x7fd8d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.614+0000 7fd8f4b52640 1 -- 172.21.15.37:0/3216768757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d8003520 con 0x7fd8d8003f60 2024-04-26T20:22:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.615+0000 7fd8f4b52640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8d8003f60 0x7fd8d8004800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009ac0 tx=0x7fd8d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdc5dbd9bacb8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.615+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4036070 con 0x7fd8d8003f60 2024-04-26T20:22:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.615+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e4031400 con 0x7fd8d8003f60 2024-04-26T20:22:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.615+0000 7fd8effff640 1 -- 172.21.15.37:0/3216768757 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4009690 con 0x7fd8d8003f60 2024-04-26T20:22:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.616+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3216768757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8d8003f60 msgr2=0x7fd8d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.616+0000 7fd8f75b7640 1 --2- 172.21.15.37:0/3216768757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8d8003f60 0x7fd8d8004800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009ac0 tx=0x7fd8d8005830 comp rx=0 tx=0).stop 2024-04-26T20:22:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.616+0000 7fd8f5b54640 1 -- 172.21.15.37:0/3216768757 reap_dead start 2024-04-26T20:22:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.616+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3216768757 shutdown_connections 2024-04-26T20:22:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.616+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3216768757 >> 172.21.15.37:0/3216768757 conn(0x7fd8f009e550 msgr2=0x7fd8f00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.616+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3216768757 shutdown_connections 2024-04-26T20:22:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.616+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3216768757 wait complete. 2024-04-26T20:22:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.617+0000 7fd8f75b7640 1 Processor -- start 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.617+0000 7fd8f75b7640 1 -- start start 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f75b7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8d8003f60 0x7fd8f0158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f75b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f00cf130 0x7fd8f00ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f75b7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f0155770 0x7fd8f00ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f75b7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f00d2e10 con 0x7fd8f00cf130 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f75b7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f00d11c0 con 0x7fd8d8003f60 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f75b7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f00d30f0 con 0x7fd8f0155770 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f4b52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f00cf130 0x7fd8f00ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f4b52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f00cf130 0x7fd8f00ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60646/0 (socket says 172.21.15.37:60646) 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f4b52640 1 -- 172.21.15.37:0/3850621969 learned_addr learned my addr 172.21.15.37:0/3850621969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f5b54640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f0155770 0x7fd8f00ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f5353640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8d8003f60 0x7fd8f0158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f4b52640 1 -- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f0155770 msgr2=0x7fd8f00ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f4b52640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f0155770 0x7fd8f00ba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f4b52640 1 -- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8d8003f60 msgr2=0x7fd8f0158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f4b52640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8d8003f60 0x7fd8f0158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f4b52640 1 -- 172.21.15.37:0/3850621969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f0159360 con 0x7fd8f00cf130 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.618+0000 7fd8f5b54640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f0155770 0x7fd8f00ba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.619+0000 7fd8f4b52640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f00cf130 0x7fd8f00ba3a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4034040 tx=0x7fd8e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.619+0000 7fd8edffb640 1 -- 172.21.15.37:0/3850621969 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4040020 con 0x7fd8f00cf130 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.619+0000 7fd8edffb640 1 -- 172.21.15.37:0/3850621969 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e4003c60 con 0x7fd8f00cf130 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.619+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3850621969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f01587e0 con 0x7fd8f00cf130 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.619+0000 7fd8edffb640 1 -- 172.21.15.37:0/3850621969 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4036040 con 0x7fd8f00cf130 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.619+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3850621969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f0158da0 con 0x7fd8f00cf130 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.620+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3850621969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f006a360 con 0x7fd8f00cf130 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.620+0000 7fd8edffb640 1 -- 172.21.15.37:0/3850621969 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8e4043020 con 0x7fd8f00cf130 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.621+0000 7fd8edffb640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd8dc06c8f0 0x7fd8dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.621+0000 7fd8edffb640 1 -- 172.21.15.37:0/3850621969 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd8e40abf70 con 0x7fd8f00cf130 2024-04-26T20:22:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.621+0000 7fd8f5353640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd8dc06c8f0 0x7fd8dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.622+0000 7fd8f5353640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd8dc06c8f0 0x7fd8dc06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8000fb0 tx=0x7fd8e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.625+0000 7fd8edffb640 1 -- 172.21.15.37:0/3850621969 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8e404a030 con 0x7fd8f00cf130 2024-04-26T20:22:25.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.810+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3850621969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd8f00d4530 con 0x7fd8f00cf130 2024-04-26T20:22:25.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.810+0000 7fd8edffb640 1 -- 172.21.15.37:0/3850621969 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fd8e40044e0 con 0x7fd8f00cf130 2024-04-26T20:22:25.811 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:25.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd8dc06c8f0 msgr2=0x7fd8dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd8dc06c8f0 0x7fd8dc06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8000fb0 tx=0x7fd8e8009040 comp rx=0 tx=0).stop 2024-04-26T20:22:25.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3850621969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f00cf130 msgr2=0x7fd8f00ba3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f00cf130 0x7fd8f00ba3a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4034040 tx=0x7fd8e4004730 comp rx=0 tx=0).stop 2024-04-26T20:22:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3850621969 shutdown_connections 2024-04-26T20:22:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd8dc06c8f0 0x7fd8dc06edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8f0155770 0x7fd8f00ba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f00cf130 0x7fd8f00ba3a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 --2- 172.21.15.37:0/3850621969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8d8003f60 0x7fd8f0158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3850621969 >> 172.21.15.37:0/3850621969 conn(0x7fd8f009e550 msgr2=0x7fd8f006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3850621969 shutdown_connections 2024-04-26T20:22:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.812+0000 7fd8f75b7640 1 -- 172.21.15.37:0/3850621969 wait complete. 2024-04-26T20:22:25.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:22:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.925+0000 7f4331746640 1 Processor -- start 2024-04-26T20:22:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.925+0000 7f4331746640 1 -- start start 2024-04-26T20:22:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.926+0000 7f4331746640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f432c15f760 0x7f432c0b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.926+0000 7f4331746640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f432c0d3de0 con 0x7f432c0d22e0 2024-04-26T20:22:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.926+0000 7f4331746640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f432c0d40c0 con 0x7f432c15f380 2024-04-26T20:22:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.926+0000 7f4331746640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f432c0d43a0 con 0x7f432c15f760 2024-04-26T20:22:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.926+0000 7f432b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f432c15f760 0x7f432c0b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.926+0000 7f4323fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f432c0d22e0 0x7f432c0b8a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57248/0 (socket says 172.21.15.37:57248) 2024-04-26T20:22:25.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.926+0000 7f4323fff640 1 -- 172.21.15.37:0/2010598206 learned_addr learned my addr 172.21.15.37:0/2010598206 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:25.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.926+0000 7f432b7fe640 1 -- 172.21.15.37:0/2010598206 >> v1:172.21.15.134:6789/0 conn(0x7f432c15f380 legacy=0x7f432c0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:25.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.926+0000 7f432b7fe640 1 -- 172.21.15.37:0/2010598206 >> v1:172.21.15.37:6789/0 conn(0x7f432c0d22e0 legacy=0x7f432c0b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.926+0000 7f432a7fc640 1 -- 172.21.15.37:0/2010598206 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2716605498 0 0) 0x7f432c0d3de0 con 0x7f432c0d22e0 2024-04-26T20:22:25.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.926+0000 7f432b7fe640 1 -- 172.21.15.37:0/2010598206 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c0b8280 con 0x7f432c15f760 2024-04-26T20:22:25.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.927+0000 7f432b7fe640 1 --2- 172.21.15.37:0/2010598206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f432c15f760 0x7f432c0b9190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f432c0d4520 tx=0x7f432402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe8e4842644add8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:25.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.927+0000 7f432a7fc640 1 -- 172.21.15.37:0/2010598206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324007740 con 0x7f432c15f760 2024-04-26T20:22:25.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.927+0000 7f432a7fc640 1 -- 172.21.15.37:0/2010598206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43240078e0 con 0x7f432c15f760 2024-04-26T20:22:25.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.927+0000 7f432a7fc640 1 -- 172.21.15.37:0/2010598206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324007cb0 con 0x7f432c15f760 2024-04-26T20:22:25.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.928+0000 7f4331746640 1 -- 172.21.15.37:0/2010598206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f432c15f760 msgr2=0x7f432c0b9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.928+0000 7f4331746640 1 --2- 172.21.15.37:0/2010598206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f432c15f760 0x7f432c0b9190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f432c0d4520 tx=0x7f432402a750 comp rx=0 tx=0).stop 2024-04-26T20:22:25.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.928+0000 7f4331746640 1 -- 172.21.15.37:0/2010598206 shutdown_connections 2024-04-26T20:22:25.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.928+0000 7f4331746640 1 --2- 172.21.15.37:0/2010598206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f432c15f760 0x7f432c0b9190 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.928+0000 7f4331746640 1 -- 172.21.15.37:0/2010598206 >> 172.21.15.37:0/2010598206 conn(0x7f432c09e560 msgr2=0x7f432c0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:25.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.928+0000 7f4331746640 1 -- 172.21.15.37:0/2010598206 shutdown_connections 2024-04-26T20:22:25.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.928+0000 7f4331746640 1 -- 172.21.15.37:0/2010598206 wait complete. 2024-04-26T20:22:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.929+0000 7f4331746640 1 Processor -- start 2024-04-26T20:22:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.929+0000 7f4331746640 1 -- start start 2024-04-26T20:22:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f4331746640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c0d22e0 0x7f432c158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f4331746640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f432c15f380 0x7f432c1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f4331746640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f432c15f760 0x7f432c0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f4331746640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0d1280 con 0x7f432c15f760 2024-04-26T20:22:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f4331746640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0d1100 con 0x7f432c0d22e0 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f4331746640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f432c0d5b50 con 0x7f432c15f380 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f432b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f432c15f760 0x7f432c0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f432b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f432c15f760 0x7f432c0cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57894/0 (socket says 172.21.15.37:57894) 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f432b7fe640 1 -- 172.21.15.37:0/3684553362 learned_addr learned my addr 172.21.15.37:0/3684553362 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f432affd640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c0d22e0 0x7f432c158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f4323fff640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f432c15f380 0x7f432c1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f432b7fe640 1 -- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f432c15f380 msgr2=0x7f432c1548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f432b7fe640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f432c15f380 0x7f432c1548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f432b7fe640 1 -- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c0d22e0 msgr2=0x7f432c158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f432b7fe640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c0d22e0 0x7f432c158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f432b7fe640 1 -- 172.21.15.37:0/3684553362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c156900 con 0x7f432c15f760 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f432b7fe640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f432c15f760 0x7f432c0cdea0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f432402ad40 tx=0x7f4324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:25.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f43237fe640 1 -- 172.21.15.37:0/3684553362 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324040070 con 0x7f432c15f760 2024-04-26T20:22:25.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f43237fe640 1 -- 172.21.15.37:0/3684553362 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4324031c80 con 0x7f432c15f760 2024-04-26T20:22:25.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f4331746640 1 -- 172.21.15.37:0/3684553362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c1612e0 con 0x7f432c15f760 2024-04-26T20:22:25.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.930+0000 7f43237fe640 1 -- 172.21.15.37:0/3684553362 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324003bb0 con 0x7f432c15f760 2024-04-26T20:22:25.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.931+0000 7f4331746640 1 -- 172.21.15.37:0/3684553362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c1618a0 con 0x7f432c15f760 2024-04-26T20:22:25.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.931+0000 7f4331746640 1 -- 172.21.15.37:0/3684553362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c06a360 con 0x7f432c15f760 2024-04-26T20:22:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.932+0000 7f43237fe640 1 -- 172.21.15.37:0/3684553362 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4324039040 con 0x7f432c15f760 2024-04-26T20:22:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.932+0000 7f43237fe640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f430c06c920 0x7f430c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.932+0000 7f43237fe640 1 -- 172.21.15.37:0/3684553362 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f43240ad120 con 0x7f432c15f760 2024-04-26T20:22:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.932+0000 7f432affd640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f430c06c920 0x7f430c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.933+0000 7f432affd640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f430c06c920 0x7f430c06ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f431c004ea0 tx=0x7f431c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:25.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:25.937+0000 7f43237fe640 1 -- 172.21.15.37:0/3684553362 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f432407b540 con 0x7f432c15f760 2024-04-26T20:22:26.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.133+0000 7f4331746640 1 -- 172.21.15.37:0/3684553362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f432c0bade0 con 0x7f432c15f760 2024-04-26T20:22:26.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.134+0000 7f43237fe640 1 -- 172.21.15.37:0/3684553362 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f432407ae40 con 0x7f432c15f760 2024-04-26T20:22:26.135 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-26T20:22:26.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 -- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f430c06c920 msgr2=0x7f430c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f430c06c920 0x7f430c06ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f431c004ea0 tx=0x7f431c009040 comp rx=0 tx=0).stop 2024-04-26T20:22:26.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 -- 172.21.15.37:0/3684553362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f432c15f760 msgr2=0x7f432c0cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f432c15f760 0x7f432c0cdea0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f432402ad40 tx=0x7f4324004a40 comp rx=0 tx=0).stop 2024-04-26T20:22:26.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 -- 172.21.15.37:0/3684553362 shutdown_connections 2024-04-26T20:22:26.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f432c15f760 0x7f432c0cdea0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f430c06c920 0x7f430c06ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f432c15f380 0x7f432c1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 --2- 172.21.15.37:0/3684553362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c0d22e0 0x7f432c158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 -- 172.21.15.37:0/3684553362 >> 172.21.15.37:0/3684553362 conn(0x7f432c09e560 msgr2=0x7f432c0cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:26.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 -- 172.21.15.37:0/3684553362 shutdown_connections 2024-04-26T20:22:26.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.136+0000 7f4331746640 1 -- 172.21.15.37:0/3684553362 wait complete. 2024-04-26T20:22:26.148 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-26T20:22:26.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:26.148 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-26T20:22:26.174 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:22:26.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:26.175 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T20:22:26.231 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:22:26.231 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:26.250 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:22:26.250 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:26.307 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:22:26.308 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:26.365 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:26.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.465+0000 7f7d45485640 1 Processor -- start 2024-04-26T20:22:26.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.465+0000 7f7d45485640 1 -- start start 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.466+0000 7f7d45485640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7d400ce860 0x7f7d400c7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.466+0000 7f7d45485640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d400ced60 con 0x7f7d4015d310 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.466+0000 7f7d45485640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d400cf040 con 0x7f7d400d1c50 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.466+0000 7f7d45485640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f7d400c6ab0 con 0x7f7d400ce860 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.466+0000 7f7d3f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7d400ce860 0x7f7d400c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.466+0000 7f7d3f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7d400ce860 0x7f7d400c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57898/0 (socket says 172.21.15.37:57898) 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.466+0000 7f7d3effd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7d4015d310 0x7f7d400d1540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57260/0 (socket says 172.21.15.37:57260) 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.466+0000 7f7d3f7fe640 1 -- 172.21.15.37:0/3776449030 learned_addr learned my addr 172.21.15.37:0/3776449030 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.467+0000 7f7d3f7fe640 1 -- 172.21.15.37:0/3776449030 >> v1:172.21.15.134:6789/0 conn(0x7f7d400d1c50 legacy=0x7f7d400c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.467+0000 7f7d3f7fe640 1 -- 172.21.15.37:0/3776449030 >> v1:172.21.15.37:6789/0 conn(0x7f7d4015d310 legacy=0x7f7d400d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.467+0000 7f7d3f7fe640 1 -- 172.21.15.37:0/3776449030 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d400cf320 con 0x7f7d400ce860 2024-04-26T20:22:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.467+0000 7f7d3dffb640 1 -- 172.21.15.37:0/3776449030 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1543326483 0 0) 0x7f7d400ced60 con 0x7f7d4015d310 2024-04-26T20:22:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.467+0000 7f7d3f7fe640 1 --2- 172.21.15.37:0/3776449030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d400ce860 0x7f7d400c7910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7d400c6c30 tx=0x7f7d3002a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c17029665324a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.467+0000 7f7d3dffb640 1 -- 172.21.15.37:0/3776449030 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30036070 con 0x7f7d400ce860 2024-04-26T20:22:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.467+0000 7f7d3dffb640 1 -- 172.21.15.37:0/3776449030 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d30031600 con 0x7f7d400ce860 2024-04-26T20:22:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.467+0000 7f7d3dffb640 1 -- 172.21.15.37:0/3776449030 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d300319d0 con 0x7f7d400ce860 2024-04-26T20:22:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.468+0000 7f7d45485640 1 -- 172.21.15.37:0/3776449030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d400ce860 msgr2=0x7f7d400c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.468+0000 7f7d45485640 1 --2- 172.21.15.37:0/3776449030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d400ce860 0x7f7d400c7910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7d400c6c30 tx=0x7f7d3002a7e0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.468+0000 7f7d45485640 1 -- 172.21.15.37:0/3776449030 shutdown_connections 2024-04-26T20:22:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.468+0000 7f7d45485640 1 --2- 172.21.15.37:0/3776449030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d400ce860 0x7f7d400c7910 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.468+0000 7f7d45485640 1 -- 172.21.15.37:0/3776449030 >> 172.21.15.37:0/3776449030 conn(0x7f7d400b4a50 msgr2=0x7f7d400b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.468+0000 7f7d45485640 1 -- 172.21.15.37:0/3776449030 shutdown_connections 2024-04-26T20:22:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.468+0000 7f7d45485640 1 -- 172.21.15.37:0/3776449030 wait complete. 2024-04-26T20:22:26.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.469+0000 7f7d45485640 1 Processor -- start 2024-04-26T20:22:26.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.469+0000 7f7d45485640 1 -- start start 2024-04-26T20:22:26.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d45485640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d400ce860 0x7f7d40151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d45485640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d400d1c50 0x7f7d40158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d45485640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d4015d310 0x7f7d40070850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d45485640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d400b5970 con 0x7f7d4015d310 2024-04-26T20:22:26.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d45485640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d400b57f0 con 0x7f7d400ce860 2024-04-26T20:22:26.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d45485640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4015d6f0 con 0x7f7d400d1c50 2024-04-26T20:22:26.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d4015d310 0x7f7d40070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d4015d310 0x7f7d40070850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57910/0 (socket says 172.21.15.37:57910) 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d3f7fe640 1 -- 172.21.15.37:0/2193500196 learned_addr learned my addr 172.21.15.37:0/2193500196 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d3effd640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d400ce860 0x7f7d40151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d3e7fc640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d400d1c50 0x7f7d40158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d3f7fe640 1 -- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d400d1c50 msgr2=0x7f7d40158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d3f7fe640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d400d1c50 0x7f7d40158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.470+0000 7f7d3f7fe640 1 -- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d400ce860 msgr2=0x7f7d40151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.471+0000 7f7d3f7fe640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d400ce860 0x7f7d40151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.471+0000 7f7d3f7fe640 1 -- 172.21.15.37:0/2193500196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d40158ba0 con 0x7f7d4015d310 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.471+0000 7f7d3e7fc640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d400d1c50 0x7f7d40158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.471+0000 7f7d3f7fe640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d4015d310 0x7f7d40070850 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009e50 tx=0x7f7d30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.471+0000 7f7d377fe640 1 -- 172.21.15.37:0/2193500196 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30036050 con 0x7f7d4015d310 2024-04-26T20:22:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.471+0000 7f7d377fe640 1 -- 172.21.15.37:0/2193500196 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d30042070 con 0x7f7d4015d310 2024-04-26T20:22:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.471+0000 7f7d45485640 1 -- 172.21.15.37:0/2193500196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d40158e50 con 0x7f7d4015d310 2024-04-26T20:22:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.471+0000 7f7d377fe640 1 -- 172.21.15.37:0/2193500196 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30009510 con 0x7f7d4015d310 2024-04-26T20:22:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.471+0000 7f7d45485640 1 -- 172.21.15.37:0/2193500196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d40159410 con 0x7f7d4015d310 2024-04-26T20:22:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.472+0000 7f7d45485640 1 -- 172.21.15.37:0/2193500196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d400693e0 con 0x7f7d4015d310 2024-04-26T20:22:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.472+0000 7f7d377fe640 1 -- 172.21.15.37:0/2193500196 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d30007bd0 con 0x7f7d4015d310 2024-04-26T20:22:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.473+0000 7f7d377fe640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7d2006c8f0 0x7f7d2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.473+0000 7f7d377fe640 1 -- 172.21.15.37:0/2193500196 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7d300acbc0 con 0x7f7d4015d310 2024-04-26T20:22:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.473+0000 7f7d3effd640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7d2006c8f0 0x7f7d2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.474+0000 7f7d3effd640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7d2006c8f0 0x7f7d2006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000f60 tx=0x7f7d38007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.477+0000 7f7d377fe640 1 -- 172.21.15.37:0/2193500196 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d30045360 con 0x7f7d4015d310 2024-04-26T20:22:26.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.665+0000 7f7d45485640 1 -- 172.21.15.37:0/2193500196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d400d1320 con 0x7f7d4015d310 2024-04-26T20:22:26.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.666+0000 7f7d377fe640 1 -- 172.21.15.37:0/2193500196 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f7d3007b020 con 0x7f7d4015d310 2024-04-26T20:22:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:26.668 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:26.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 -- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7d2006c8f0 msgr2=0x7f7d2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7d2006c8f0 0x7f7d2006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000f60 tx=0x7f7d38007d00 comp rx=0 tx=0).stop 2024-04-26T20:22:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 -- 172.21.15.37:0/2193500196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d4015d310 msgr2=0x7f7d40070850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d4015d310 0x7f7d40070850 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009e50 tx=0x7f7d30004a40 comp rx=0 tx=0).stop 2024-04-26T20:22:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 -- 172.21.15.37:0/2193500196 shutdown_connections 2024-04-26T20:22:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d4015d310 0x7f7d40070850 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7d2006c8f0 0x7f7d2006edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d400d1c50 0x7f7d40158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 --2- 172.21.15.37:0/2193500196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d400ce860 0x7f7d40151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 -- 172.21.15.37:0/2193500196 >> 172.21.15.37:0/2193500196 conn(0x7f7d400b4a50 msgr2=0x7f7d400cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 -- 172.21.15.37:0/2193500196 shutdown_connections 2024-04-26T20:22:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.668+0000 7f7d45485640 1 -- 172.21.15.37:0/2193500196 wait complete. 2024-04-26T20:22:26.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:26.681 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:26.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.784+0000 7f2cfee69640 1 Processor -- start 2024-04-26T20:22:26.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.784+0000 7f2cfee69640 1 -- start start 2024-04-26T20:22:26.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfee69640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2cf80b2290 0x7f2cf80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfee69640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2cf80d26f0 0x7f2cf80cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfee69640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cf80cd640 con 0x7f2cf80cf870 2024-04-26T20:22:26.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfee69640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2cf80d2b20 con 0x7f2cf80b2290 2024-04-26T20:22:26.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfee69640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2cf80cd920 con 0x7f2cf80d26f0 2024-04-26T20:22:26.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfcc05640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2cf80b2290 0x7f2cf80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cf7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2cf80cf870 0x7f2cf80d1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57270/0 (socket says 172.21.15.37:57270) 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cf7fff640 1 -- 172.21.15.37:0/3672461884 learned_addr learned my addr 172.21.15.37:0/3672461884 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfcc05640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2cf80b2290 0x7f2cf80b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57920/0 (socket says 172.21.15.37:57920) 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfd406640 1 --2- 172.21.15.37:0/3672461884 >> v2:172.21.15.134:3300/0 conn(0x7f2cf80d26f0 0x7f2cf80cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfcc05640 1 -- 172.21.15.37:0/3672461884 >> v2:172.21.15.134:3300/0 conn(0x7f2cf80d26f0 msgr2=0x7f2cf80cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfcc05640 1 --2- 172.21.15.37:0/3672461884 >> v2:172.21.15.134:3300/0 conn(0x7f2cf80d26f0 0x7f2cf80cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfcc05640 1 -- 172.21.15.37:0/3672461884 >> v1:172.21.15.37:6789/0 conn(0x7f2cf80cf870 legacy=0x7f2cf80d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cf77fe640 1 -- 172.21.15.37:0/3672461884 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 398447775 0 0) 0x7f2cf80cd640 con 0x7f2cf80cf870 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfcc05640 1 -- 172.21.15.37:0/3672461884 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf80cdc00 con 0x7f2cf80b2290 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.785+0000 7f2cfd406640 1 --2- 172.21.15.37:0/3672461884 >> v2:172.21.15.134:3300/0 conn(0x7f2cf80d26f0 0x7f2cf80cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.786+0000 7f2cfcc05640 1 --2- 172.21.15.37:0/3672461884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf80b2290 0x7f2cf80b2930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8073b10 tx=0x7f2cec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef219365ce2594e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.786+0000 7f2cf77fe640 1 -- 172.21.15.37:0/3672461884 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec037070 con 0x7f2cf80b2290 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.786+0000 7f2cf77fe640 1 -- 172.21.15.37:0/3672461884 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cec032440 con 0x7f2cf80b2290 2024-04-26T20:22:26.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.786+0000 7f2cf77fe640 1 -- 172.21.15.37:0/3672461884 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec032810 con 0x7f2cf80b2290 2024-04-26T20:22:26.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.787+0000 7f2cfee69640 1 -- 172.21.15.37:0/3672461884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf80b2290 msgr2=0x7f2cf80b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.787+0000 7f2cfee69640 1 --2- 172.21.15.37:0/3672461884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf80b2290 0x7f2cf80b2930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8073b10 tx=0x7f2cec02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.787+0000 7f2cfee69640 1 -- 172.21.15.37:0/3672461884 shutdown_connections 2024-04-26T20:22:26.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.787+0000 7f2cfee69640 1 --2- 172.21.15.37:0/3672461884 >> v2:172.21.15.134:3300/0 conn(0x7f2cf80d26f0 0x7f2cf80cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.787+0000 7f2cfee69640 1 --2- 172.21.15.37:0/3672461884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf80b2290 0x7f2cf80b2930 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.787+0000 7f2cfee69640 1 -- 172.21.15.37:0/3672461884 >> 172.21.15.37:0/3672461884 conn(0x7f2cf80d2ff0 msgr2=0x7f2cf80d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:26.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.787+0000 7f2cfee69640 1 -- 172.21.15.37:0/3672461884 shutdown_connections 2024-04-26T20:22:26.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.787+0000 7f2cfee69640 1 -- 172.21.15.37:0/3672461884 wait complete. 2024-04-26T20:22:26.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.788+0000 7f2cfee69640 1 Processor -- start 2024-04-26T20:22:26.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.788+0000 7f2cfee69640 1 -- start start 2024-04-26T20:22:26.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cfee69640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cf80b2290 0x7f2cf81f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cfee69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf80cf870 0x7f2cf815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cfee69640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cf80d26f0 0x7f2cf815de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cfee69640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf80b2670 con 0x7f2cf80cf870 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cfee69640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf80ce1c0 con 0x7f2cf80d26f0 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cfee69640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf815ec60 con 0x7f2cf80b2290 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cf7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf80cf870 0x7f2cf815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cf7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf80cf870 0x7f2cf815d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57924/0 (socket says 172.21.15.37:57924) 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cf7fff640 1 -- 172.21.15.37:0/3286346275 learned_addr learned my addr 172.21.15.37:0/3286346275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cfd406640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cf80d26f0 0x7f2cf815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cfcc05640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cf80b2290 0x7f2cf81f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cf7fff640 1 -- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cf80b2290 msgr2=0x7f2cf81f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cf7fff640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cf80b2290 0x7f2cf81f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cf7fff640 1 -- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cf80d26f0 msgr2=0x7f2cf815de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cf7fff640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cf80d26f0 0x7f2cf815de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.789+0000 7f2cf7fff640 1 -- 172.21.15.37:0/3286346275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf81f7f00 con 0x7f2cf80cf870 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.790+0000 7f2cf7fff640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf80cf870 0x7f2cf815d8c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400c4d0 tx=0x7f2ce400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.790+0000 7f2cf57fa640 1 -- 172.21.15.37:0/3286346275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4018070 con 0x7f2cf80cf870 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.790+0000 7f2cf57fa640 1 -- 172.21.15.37:0/3286346275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce4007500 con 0x7f2cf80cf870 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.790+0000 7f2cfee69640 1 -- 172.21.15.37:0/3286346275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf81f81a0 con 0x7f2cf80cf870 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.790+0000 7f2cf57fa640 1 -- 172.21.15.37:0/3286346275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4004a20 con 0x7f2cf80cf870 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.790+0000 7f2cfee69640 1 -- 172.21.15.37:0/3286346275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf81f8760 con 0x7f2cf80cf870 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.791+0000 7f2cfee69640 1 -- 172.21.15.37:0/3286346275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf806a360 con 0x7f2cf80cf870 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.791+0000 7f2cf57fa640 1 -- 172.21.15.37:0/3286346275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ce401b070 con 0x7f2cf80cf870 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.792+0000 7f2cf57fa640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2cdc06c920 0x7f2cdc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.792+0000 7f2cf57fa640 1 -- 172.21.15.37:0/3286346275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ce408ae00 con 0x7f2cf80cf870 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.792+0000 7f2cfcc05640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2cdc06c920 0x7f2cdc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.793+0000 7f2cfcc05640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2cdc06c920 0x7f2cdc06ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2cec034000 tx=0x7f2cec02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:26.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.796+0000 7f2cf57fa640 1 -- 172.21.15.37:0/3286346275 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2ce40592c0 con 0x7f2cf80cf870 2024-04-26T20:22:26.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.979+0000 7f2cfee69640 1 -- 172.21.15.37:0/3286346275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2cf81572e0 con 0x7f2cf80cf870 2024-04-26T20:22:26.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.979+0000 7f2cf57fa640 1 -- 172.21.15.37:0/3286346275 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f2ce4058bc0 con 0x7f2cf80cf870 2024-04-26T20:22:26.980 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 -- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2cdc06c920 msgr2=0x7f2cdc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2cdc06c920 0x7f2cdc06ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2cec034000 tx=0x7f2cec02e000 comp rx=0 tx=0).stop 2024-04-26T20:22:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 -- 172.21.15.37:0/3286346275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf80cf870 msgr2=0x7f2cf815d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf80cf870 0x7f2cf815d8c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400c4d0 tx=0x7f2ce400c990 comp rx=0 tx=0).stop 2024-04-26T20:22:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 -- 172.21.15.37:0/3286346275 shutdown_connections 2024-04-26T20:22:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cf80d26f0 0x7f2cf815de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf80cf870 0x7f2cf815d8c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2cdc06c920 0x7f2cdc06ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 --2- 172.21.15.37:0/3286346275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cf80b2290 0x7f2cf81f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 -- 172.21.15.37:0/3286346275 >> 172.21.15.37:0/3286346275 conn(0x7f2cf80d2ff0 msgr2=0x7f2cf80d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:26.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 -- 172.21.15.37:0/3286346275 shutdown_connections 2024-04-26T20:22:26.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:26.982+0000 7f2cfee69640 1 -- 172.21.15.37:0/3286346275 wait complete. 2024-04-26T20:22:26.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:27.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.096+0000 7f60443a4640 1 Processor -- start 2024-04-26T20:22:27.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.096+0000 7f60443a4640 1 -- start start 2024-04-26T20:22:27.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.097+0000 7f60443a4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f603c1610c0 0x7f603c0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.097+0000 7f60443a4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f603c0d2150 0x7f603c0d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.097+0000 7f60443a4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f603c161510 con 0x7f603c160800 2024-04-26T20:22:27.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.097+0000 7f60443a4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f603c0d25c0 con 0x7f603c0d2150 2024-04-26T20:22:27.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.097+0000 7f60443a4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f603c0d0930 con 0x7f603c1610c0 2024-04-26T20:22:27.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.097+0000 7f6042941640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f603c0d2150 0x7f603c0d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.097+0000 7f6042941640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f603c0d2150 0x7f603c0d3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57928/0 (socket says 172.21.15.37:57928) 2024-04-26T20:22:27.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.097+0000 7f6042941640 1 -- 172.21.15.37:0/1675234871 learned_addr learned my addr 172.21.15.37:0/1675234871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:27.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.098+0000 7f604193f640 1 --2- 172.21.15.37:0/1675234871 >> v2:172.21.15.134:3300/0 conn(0x7f603c1610c0 0x7f603c0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.098+0000 7f6042941640 1 -- 172.21.15.37:0/1675234871 >> v2:172.21.15.134:3300/0 conn(0x7f603c1610c0 msgr2=0x7f603c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:27.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.098+0000 7f6042941640 1 --2- 172.21.15.37:0/1675234871 >> v2:172.21.15.134:3300/0 conn(0x7f603c1610c0 0x7f603c0d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.098+0000 7f6042941640 1 -- 172.21.15.37:0/1675234871 >> v1:172.21.15.134:6789/0 conn(0x7f603c160800 legacy=0x7f603c0d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.098+0000 7f6042941640 1 -- 172.21.15.37:0/1675234871 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c0d0c00 con 0x7f603c0d2150 2024-04-26T20:22:27.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.098+0000 7f6042941640 1 --2- 172.21.15.37:0/1675234871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f603c0d2150 0x7f603c0d3610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f603c0b6090 tx=0x7f602c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359eaff2b3e8ef8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:27.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.099+0000 7f604113e640 1 -- 172.21.15.37:0/1675234871 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c037070 con 0x7f603c0d2150 2024-04-26T20:22:27.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.099+0000 7f604113e640 1 -- 172.21.15.37:0/1675234871 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f602c032440 con 0x7f603c0d2150 2024-04-26T20:22:27.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.099+0000 7f604113e640 1 -- 172.21.15.37:0/1675234871 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c032810 con 0x7f603c0d2150 2024-04-26T20:22:27.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.099+0000 7f60443a4640 1 -- 172.21.15.37:0/1675234871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f603c0d2150 msgr2=0x7f603c0d3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.099+0000 7f60443a4640 1 --2- 172.21.15.37:0/1675234871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f603c0d2150 0x7f603c0d3610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f603c0b6090 tx=0x7f602c02a270 comp rx=0 tx=0).stop 2024-04-26T20:22:27.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.099+0000 7f60443a4640 1 -- 172.21.15.37:0/1675234871 shutdown_connections 2024-04-26T20:22:27.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.099+0000 7f60443a4640 1 --2- 172.21.15.37:0/1675234871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f603c0d2150 0x7f603c0d3610 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.099+0000 7f60443a4640 1 --2- 172.21.15.37:0/1675234871 >> v2:172.21.15.134:3300/0 conn(0x7f603c1610c0 0x7f603c0d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.099+0000 7f60443a4640 1 -- 172.21.15.37:0/1675234871 >> 172.21.15.37:0/1675234871 conn(0x7f603c09e550 msgr2=0x7f603c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:27.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.099+0000 7f60443a4640 1 -- 172.21.15.37:0/1675234871 shutdown_connections 2024-04-26T20:22:27.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.099+0000 7f60443a4640 1 -- 172.21.15.37:0/1675234871 wait complete. 2024-04-26T20:22:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.100+0000 7f60443a4640 1 Processor -- start 2024-04-26T20:22:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.100+0000 7f60443a4640 1 -- start start 2024-04-26T20:22:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.101+0000 7f60443a4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f603c0d2150 0x7f603c1f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.101+0000 7f60443a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f603c160800 0x7f603c157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.101+0000 7f60443a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f603c1610c0 0x7f603c0b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.101+0000 7f60443a4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f603c0d2e10 con 0x7f603c160800 2024-04-26T20:22:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.101+0000 7f60443a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f603c0d11c0 con 0x7f603c1610c0 2024-04-26T20:22:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.101+0000 7f60443a4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f603c0d30f0 con 0x7f603c0d2150 2024-04-26T20:22:27.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.101+0000 7f604193f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f603c160800 0x7f603c157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.101+0000 7f604193f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f603c160800 0x7f603c157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57936/0 (socket says 172.21.15.37:57936) 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.101+0000 7f604193f640 1 -- 172.21.15.37:0/258192171 learned_addr learned my addr 172.21.15.37:0/258192171 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.101+0000 7f6042941640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f603c1610c0 0x7f603c0b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.101+0000 7f6042140640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f603c0d2150 0x7f603c1f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f604193f640 1 -- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f603c0d2150 msgr2=0x7f603c1f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f604193f640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f603c0d2150 0x7f603c1f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f604193f640 1 -- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f603c1610c0 msgr2=0x7f603c0b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f604193f640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f603c1610c0 0x7f603c0b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f604193f640 1 -- 172.21.15.37:0/258192171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c0b5bf0 con 0x7f603c160800 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f6042941640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f603c1610c0 0x7f603c0b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f604193f640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f603c160800 0x7f603c157340 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f603000d600 tx=0x7f603000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f603affd640 1 -- 172.21.15.37:0/258192171 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300125f0 con 0x7f603c160800 2024-04-26T20:22:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f603affd640 1 -- 172.21.15.37:0/258192171 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6030019070 con 0x7f603c160800 2024-04-26T20:22:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f60443a4640 1 -- 172.21.15.37:0/258192171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c0ce670 con 0x7f603c160800 2024-04-26T20:22:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f603affd640 1 -- 172.21.15.37:0/258192171 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300129f0 con 0x7f603c160800 2024-04-26T20:22:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.102+0000 7f60443a4640 1 -- 172.21.15.37:0/258192171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c0cec30 con 0x7f603c160800 2024-04-26T20:22:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.103+0000 7f60443a4640 1 -- 172.21.15.37:0/258192171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603c06a360 con 0x7f603c160800 2024-04-26T20:22:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.103+0000 7f603affd640 1 -- 172.21.15.37:0/258192171 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6030012b90 con 0x7f603c160800 2024-04-26T20:22:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.104+0000 7f603affd640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f602806c8f0 0x7f602806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.104+0000 7f603affd640 1 -- 172.21.15.37:0/258192171 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f603008bfa0 con 0x7f603c160800 2024-04-26T20:22:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.104+0000 7f6042140640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f602806c8f0 0x7f602806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.105+0000 7f6042140640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f602806c8f0 0x7f602806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6034001d40 tx=0x7f6034009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:27.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.108+0000 7f603affd640 1 -- 172.21.15.37:0/258192171 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f603005a3d0 con 0x7f603c160800 2024-04-26T20:22:27.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.294+0000 7f60443a4640 1 -- 172.21.15.37:0/258192171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f603c0d2960 con 0x7f603c160800 2024-04-26T20:22:27.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.295+0000 7f603affd640 1 -- 172.21.15.37:0/258192171 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f6030059cd0 con 0x7f603c160800 2024-04-26T20:22:27.300 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:27.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 -- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f602806c8f0 msgr2=0x7f602806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f602806c8f0 0x7f602806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6034001d40 tx=0x7f6034009040 comp rx=0 tx=0).stop 2024-04-26T20:22:27.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 -- 172.21.15.37:0/258192171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f603c160800 msgr2=0x7f603c157340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f603c160800 0x7f603c157340 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f603000d600 tx=0x7f603000dac0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 -- 172.21.15.37:0/258192171 shutdown_connections 2024-04-26T20:22:27.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f603c1610c0 0x7f603c0b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f603c160800 0x7f603c157340 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f602806c8f0 0x7f602806edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 --2- 172.21.15.37:0/258192171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f603c0d2150 0x7f603c1f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 -- 172.21.15.37:0/258192171 >> 172.21.15.37:0/258192171 conn(0x7f603c09e550 msgr2=0x7f603c0d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:27.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 -- 172.21.15.37:0/258192171 shutdown_connections 2024-04-26T20:22:27.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.297+0000 7f60443a4640 1 -- 172.21.15.37:0/258192171 wait complete. 2024-04-26T20:22:27.309 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.411+0000 7fd8a9820640 1 Processor -- start 2024-04-26T20:22:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.412+0000 7fd8a9820640 1 -- start start 2024-04-26T20:22:27.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.412+0000 7fd8a9820640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 0x7fd8a40cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.412+0000 7fd8a9820640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd8a415d630 0x7fd8a415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.412+0000 7fd8a9820640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a4073080 con 0x7fd8a406c240 2024-04-26T20:22:27.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.412+0000 7fd8a9820640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd8a40d07f0 con 0x7fd8a415d630 2024-04-26T20:22:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.412+0000 7fd8a9820640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a4073340 con 0x7fd8a40cfac0 2024-04-26T20:22:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a27fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 0x7fd8a40cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a37fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd8a415d630 0x7fd8a415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a2ffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd8a406c240 0x7fd8a406c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:41014/0 (socket says 172.21.15.37:41014) 2024-04-26T20:22:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a2ffd640 1 -- 172.21.15.37:0/674892921 learned_addr learned my addr 172.21.15.37:0/674892921 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a27fc640 1 --2- 172.21.15.37:0/674892921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 0x7fd8a40cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:53676/0 (socket says 172.21.15.37:53676) 2024-04-26T20:22:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a1ffb640 1 -- 172.21.15.37:0/674892921 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3532849923 0 0) 0x7fd8a4073080 con 0x7fd8a406c240 2024-04-26T20:22:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a1ffb640 1 -- 172.21.15.37:0/674892921 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd88c003520 con 0x7fd8a406c240 2024-04-26T20:22:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a27fc640 1 -- 172.21.15.37:0/674892921 >> v2:172.21.15.134:3300/0 conn(0x7fd8a415d630 msgr2=0x7fd8a415dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a27fc640 1 --2- 172.21.15.37:0/674892921 >> v2:172.21.15.134:3300/0 conn(0x7fd8a415d630 0x7fd8a415dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a27fc640 1 -- 172.21.15.37:0/674892921 >> v1:172.21.15.134:6789/0 conn(0x7fd8a406c240 legacy=0x7fd8a406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a27fc640 1 -- 172.21.15.37:0/674892921 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a4073620 con 0x7fd8a40cfac0 2024-04-26T20:22:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.413+0000 7fd8a37fe640 1 --2- 172.21.15.37:0/674892921 >> v2:172.21.15.134:3300/0 conn(0x7fd8a415d630 0x7fd8a415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.414+0000 7fd8a27fc640 1 --2- 172.21.15.37:0/674892921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 0x7fd8a40cff20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd8980097d0 tx=0x7fd89802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d06b5c492f51bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.414+0000 7fd8a1ffb640 1 -- 172.21.15.37:0/674892921 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898037070 con 0x7fd8a40cfac0 2024-04-26T20:22:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.414+0000 7fd8a1ffb640 1 -- 172.21.15.37:0/674892921 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd898032440 con 0x7fd8a40cfac0 2024-04-26T20:22:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.414+0000 7fd8a1ffb640 1 -- 172.21.15.37:0/674892921 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898032810 con 0x7fd8a40cfac0 2024-04-26T20:22:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.415+0000 7fd8a9820640 1 -- 172.21.15.37:0/674892921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 msgr2=0x7fd8a40cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.415+0000 7fd8a9820640 1 --2- 172.21.15.37:0/674892921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 0x7fd8a40cff20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd8980097d0 tx=0x7fd89802a1c0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.415+0000 7fd8a9820640 1 -- 172.21.15.37:0/674892921 shutdown_connections 2024-04-26T20:22:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.415+0000 7fd8a9820640 1 --2- 172.21.15.37:0/674892921 >> v2:172.21.15.134:3300/0 conn(0x7fd8a415d630 0x7fd8a415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.415+0000 7fd8a9820640 1 --2- 172.21.15.37:0/674892921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 0x7fd8a40cff20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.415+0000 7fd8a9820640 1 -- 172.21.15.37:0/674892921 >> 172.21.15.37:0/674892921 conn(0x7fd8a40d1e80 msgr2=0x7fd8a40d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.415+0000 7fd8a9820640 1 -- 172.21.15.37:0/674892921 shutdown_connections 2024-04-26T20:22:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.415+0000 7fd8a9820640 1 -- 172.21.15.37:0/674892921 wait complete. 2024-04-26T20:22:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.416+0000 7fd8a9820640 1 Processor -- start 2024-04-26T20:22:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.416+0000 7fd8a9820640 1 -- start start 2024-04-26T20:22:27.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a9820640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a406c240 0x7fd8a4157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a9820640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 0x7fd8a40cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a9820640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a415d630 0x7fd8a40cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a9820640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a4073e20 con 0x7fd8a415d630 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a9820640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a40d0460 con 0x7fd8a406c240 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a9820640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a4073fa0 con 0x7fd8a40cfac0 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a415d630 0x7fd8a40cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a415d630 0x7fd8a40cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57952/0 (socket says 172.21.15.37:57952) 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a37fe640 1 -- 172.21.15.37:0/566391912 learned_addr learned my addr 172.21.15.37:0/566391912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a27fc640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 0x7fd8a40cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a2ffd640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a406c240 0x7fd8a4157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a37fe640 1 -- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 msgr2=0x7fd8a40cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a37fe640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 0x7fd8a40cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a37fe640 1 -- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a406c240 msgr2=0x7fd8a4157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a37fe640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a406c240 0x7fd8a4157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a37fe640 1 -- 172.21.15.37:0/566391912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a40c9ed0 con 0x7fd8a415d630 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd8a37fe640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a415d630 0x7fd8a40cb910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd894009f00 tx=0x7fd89400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.417+0000 7fd88bfff640 1 -- 172.21.15.37:0/566391912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894018030 con 0x7fd8a415d630 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.418+0000 7fd88bfff640 1 -- 172.21.15.37:0/566391912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89400ce50 con 0x7fd8a415d630 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.418+0000 7fd88bfff640 1 -- 172.21.15.37:0/566391912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894003bb0 con 0x7fd8a415d630 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.418+0000 7fd8a9820640 1 -- 172.21.15.37:0/566391912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a40ca1b0 con 0x7fd8a415d630 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.418+0000 7fd8a9820640 1 -- 172.21.15.37:0/566391912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a40ca770 con 0x7fd8a415d630 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.419+0000 7fd8a9820640 1 -- 172.21.15.37:0/566391912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a40693e0 con 0x7fd8a415d630 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.419+0000 7fd88bfff640 1 -- 172.21.15.37:0/566391912 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd89401b070 con 0x7fd8a415d630 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.419+0000 7fd88bfff640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd89006c8f0 0x7fd89006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.419+0000 7fd88bfff640 1 -- 172.21.15.37:0/566391912 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd89408c660 con 0x7fd8a415d630 2024-04-26T20:22:27.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.420+0000 7fd8a2ffd640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd89006c8f0 0x7fd89006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.420+0000 7fd8a2ffd640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd89006c8f0 0x7fd89006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd89c005880 tx=0x7fd89c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:27.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.424+0000 7fd88bfff640 1 -- 172.21.15.37:0/566391912 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd894010040 con 0x7fd8a415d630 2024-04-26T20:22:27.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.613+0000 7fd8a9820640 1 -- 172.21.15.37:0/566391912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd8a406ce40 con 0x7fd8a415d630 2024-04-26T20:22:27.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.613+0000 7fd88bfff640 1 -- 172.21.15.37:0/566391912 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fd894012340 con 0x7fd8a415d630 2024-04-26T20:22:27.615 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.615+0000 7fd8a9820640 1 -- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd89006c8f0 msgr2=0x7fd89006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.615+0000 7fd8a9820640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd89006c8f0 0x7fd89006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd89c005880 tx=0x7fd89c009040 comp rx=0 tx=0).stop 2024-04-26T20:22:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.615+0000 7fd8a9820640 1 -- 172.21.15.37:0/566391912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a415d630 msgr2=0x7fd8a40cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.615+0000 7fd8a9820640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a415d630 0x7fd8a40cb910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd894009f00 tx=0x7fd89400c490 comp rx=0 tx=0).stop 2024-04-26T20:22:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.615+0000 7fd8a9820640 1 -- 172.21.15.37:0/566391912 shutdown_connections 2024-04-26T20:22:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.615+0000 7fd8a9820640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a415d630 0x7fd8a40cb910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.615+0000 7fd8a9820640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd89006c8f0 0x7fd89006edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.615+0000 7fd8a9820640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8a40cfac0 0x7fd8a40cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.616+0000 7fd8a9820640 1 --2- 172.21.15.37:0/566391912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a406c240 0x7fd8a4157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.616+0000 7fd8a9820640 1 -- 172.21.15.37:0/566391912 >> 172.21.15.37:0/566391912 conn(0x7fd8a40d1e80 msgr2=0x7fd8a40ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.616+0000 7fd8a9820640 1 -- 172.21.15.37:0/566391912 shutdown_connections 2024-04-26T20:22:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.616+0000 7fd8a9820640 1 -- 172.21.15.37:0/566391912 wait complete. 2024-04-26T20:22:27.628 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:22:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.728+0000 7f456ec93640 1 Processor -- start 2024-04-26T20:22:27.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.728+0000 7f456ec93640 1 -- start start 2024-04-26T20:22:27.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f456ec93640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f45680d3820 0x7f45680d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f456ec93640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45680b4ba0 0x7f45680b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f456ec93640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45680d0210 con 0x7f4568162b40 2024-04-26T20:22:27.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f456ec93640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f45680d41c0 con 0x7f45680d3820 2024-04-26T20:22:27.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f456ec93640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f45680d04f0 con 0x7f45680b4ba0 2024-04-26T20:22:27.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f456d230640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45680b4ba0 0x7f45680b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f4567fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f45680d3820 0x7f45680d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f456d230640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45680b4ba0 0x7f45680b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:53686/0 (socket says 172.21.15.37:53686) 2024-04-26T20:22:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f456ca2f640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f4568162b40 0x7f45680d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:41026/0 (socket says 172.21.15.37:41026) 2024-04-26T20:22:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f456d230640 1 -- 172.21.15.37:0/476622084 learned_addr learned my addr 172.21.15.37:0/476622084 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f45677fe640 1 -- 172.21.15.37:0/476622084 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2177659238 0 0) 0x7f45680d0210 con 0x7f4568162b40 2024-04-26T20:22:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f456d230640 1 -- 172.21.15.37:0/476622084 >> v2:172.21.15.134:3300/0 conn(0x7f45680d3820 msgr2=0x7f45680d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.729+0000 7f456d230640 1 --2- 172.21.15.37:0/476622084 >> v2:172.21.15.134:3300/0 conn(0x7f45680d3820 0x7f45680d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.730+0000 7f456d230640 1 -- 172.21.15.37:0/476622084 >> v1:172.21.15.134:6789/0 conn(0x7f4568162b40 legacy=0x7f45680d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.730+0000 7f456d230640 1 -- 172.21.15.37:0/476622084 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45680d07d0 con 0x7f45680b4ba0 2024-04-26T20:22:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.730+0000 7f456d230640 1 --2- 172.21.15.37:0/476622084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45680b4ba0 0x7f45680b5030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4560009e50 tx=0x7f456002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50494136b18d1634 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:27.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.730+0000 7f45677fe640 1 -- 172.21.15.37:0/476622084 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560036070 con 0x7f45680b4ba0 2024-04-26T20:22:27.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.730+0000 7f45677fe640 1 -- 172.21.15.37:0/476622084 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4560031600 con 0x7f45680b4ba0 2024-04-26T20:22:27.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.731+0000 7f45677fe640 1 -- 172.21.15.37:0/476622084 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45600319d0 con 0x7f45680b4ba0 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.731+0000 7f456ec93640 1 -- 172.21.15.37:0/476622084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45680b4ba0 msgr2=0x7f45680b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.731+0000 7f456ec93640 1 --2- 172.21.15.37:0/476622084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45680b4ba0 0x7f45680b5030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4560009e50 tx=0x7f456002a710 comp rx=0 tx=0).stop 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.731+0000 7f456ec93640 1 -- 172.21.15.37:0/476622084 shutdown_connections 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.731+0000 7f456ec93640 1 --2- 172.21.15.37:0/476622084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45680b4ba0 0x7f45680b5030 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.731+0000 7f456ec93640 1 --2- 172.21.15.37:0/476622084 >> v2:172.21.15.134:3300/0 conn(0x7f45680d3820 0x7f45680d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.731+0000 7f456ec93640 1 -- 172.21.15.37:0/476622084 >> 172.21.15.37:0/476622084 conn(0x7f456809e550 msgr2=0x7f45680d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.731+0000 7f456ec93640 1 -- 172.21.15.37:0/476622084 shutdown_connections 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.731+0000 7f456ec93640 1 -- 172.21.15.37:0/476622084 wait complete. 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.732+0000 7f456ec93640 1 Processor -- start 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.732+0000 7f456ec93640 1 -- start start 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.733+0000 7f456ec93640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680b4ba0 0x7f456815d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.733+0000 7f456ec93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45680d3820 0x7f45680b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.733+0000 7f456ec93640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4568162b40 0x7f45680b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.733+0000 7f456ec93640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45680d12b0 con 0x7f45680b4ba0 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.733+0000 7f456ec93640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45680d0d90 con 0x7f45680d3820 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.733+0000 7f456ec93640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f45680cfe30 con 0x7f4568162b40 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.733+0000 7f456ca2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680b4ba0 0x7f456815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.733+0000 7f456ca2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680b4ba0 0x7f456815d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57956/0 (socket says 172.21.15.37:57956) 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.733+0000 7f456ca2f640 1 -- 172.21.15.37:0/2221420766 learned_addr learned my addr 172.21.15.37:0/2221420766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.733+0000 7f4567fff640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45680d3820 0x7f45680b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.733+0000 7f456d230640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4568162b40 0x7f45680b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.734+0000 7f456ca2f640 1 -- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4568162b40 msgr2=0x7f45680b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.734+0000 7f456ca2f640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4568162b40 0x7f45680b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.734+0000 7f456ca2f640 1 -- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45680d3820 msgr2=0x7f45680b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.734+0000 7f456ca2f640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45680d3820 0x7f45680b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.734+0000 7f456ca2f640 1 -- 172.21.15.37:0/2221420766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45680ce040 con 0x7f45680b4ba0 2024-04-26T20:22:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.734+0000 7f456ca2f640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680b4ba0 0x7f456815d880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f455c00c4d0 tx=0x7f455c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.734+0000 7f45657fa640 1 -- 172.21.15.37:0/2221420766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c018070 con 0x7f45680b4ba0 2024-04-26T20:22:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.734+0000 7f45657fa640 1 -- 172.21.15.37:0/2221420766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c007500 con 0x7f45680b4ba0 2024-04-26T20:22:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.734+0000 7f456ec93640 1 -- 172.21.15.37:0/2221420766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45680b89b0 con 0x7f45680b4ba0 2024-04-26T20:22:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.734+0000 7f45657fa640 1 -- 172.21.15.37:0/2221420766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c004a20 con 0x7f45680b4ba0 2024-04-26T20:22:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.734+0000 7f456ec93640 1 -- 172.21.15.37:0/2221420766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45680b8f70 con 0x7f45680b4ba0 2024-04-26T20:22:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.735+0000 7f456ec93640 1 -- 172.21.15.37:0/2221420766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45680b9690 con 0x7f45680b4ba0 2024-04-26T20:22:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.735+0000 7f45657fa640 1 -- 172.21.15.37:0/2221420766 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f455c01b070 con 0x7f45680b4ba0 2024-04-26T20:22:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.736+0000 7f45657fa640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f454c06c8f0 0x7f454c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.736+0000 7f45657fa640 1 -- 172.21.15.37:0/2221420766 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f455c08ae80 con 0x7f45680b4ba0 2024-04-26T20:22:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.736+0000 7f4567fff640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f454c06c8f0 0x7f454c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.737+0000 7f4567fff640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f454c06c8f0 0x7f454c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f45681f8210 tx=0x7f4554008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.740+0000 7f45657fa640 1 -- 172.21.15.37:0/2221420766 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f455c0592b0 con 0x7f45680b4ba0 2024-04-26T20:22:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.940+0000 7f456ec93640 1 -- 172.21.15.37:0/2221420766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f456806a360 con 0x7f45680b4ba0 2024-04-26T20:22:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.940+0000 7f45657fa640 1 -- 172.21.15.37:0/2221420766 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f455c058bb0 con 0x7f45680b4ba0 2024-04-26T20:22:27.942 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-26T20:22:27.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.942+0000 7f456ec93640 1 -- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f454c06c8f0 msgr2=0x7f454c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.942+0000 7f456ec93640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f454c06c8f0 0x7f454c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f45681f8210 tx=0x7f4554008040 comp rx=0 tx=0).stop 2024-04-26T20:22:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.942+0000 7f456ec93640 1 -- 172.21.15.37:0/2221420766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680b4ba0 msgr2=0x7f456815d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.942+0000 7f456ec93640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680b4ba0 0x7f456815d880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f455c00c4d0 tx=0x7f455c00c990 comp rx=0 tx=0).stop 2024-04-26T20:22:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.942+0000 7f456ec93640 1 -- 172.21.15.37:0/2221420766 shutdown_connections 2024-04-26T20:22:27.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.942+0000 7f456ec93640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f454c06c8f0 0x7f454c06edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.942+0000 7f456ec93640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4568162b40 0x7f45680b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.942+0000 7f456ec93640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45680d3820 0x7f45680b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.942+0000 7f456ec93640 1 --2- 172.21.15.37:0/2221420766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680b4ba0 0x7f456815d880 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.942+0000 7f456ec93640 1 -- 172.21.15.37:0/2221420766 >> 172.21.15.37:0/2221420766 conn(0x7f456809e550 msgr2=0x7f456806e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.943+0000 7f456ec93640 1 -- 172.21.15.37:0/2221420766 shutdown_connections 2024-04-26T20:22:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:27.943+0000 7f456ec93640 1 -- 172.21.15.37:0/2221420766 wait complete. 2024-04-26T20:22:27.954 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-26T20:22:27.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:27.955 DEBUG:teuthology.orchestra.run.smithi134:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-26T20:22:27.985 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:22:27.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:27.985 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T20:22:28.043 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T20:22:28.043 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:22:28.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.143+0000 7f6c7e693640 1 Processor -- start 2024-04-26T20:22:28.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.143+0000 7f6c7e693640 1 -- start start 2024-04-26T20:22:28.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.144+0000 7f6c7e693640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6c7806f790 0x7f6c78074030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.144+0000 7f6c7e693640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c78160d70 0x7f6c78161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.144+0000 7f6c7e693640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c78161b80 con 0x7f6c7815d780 2024-04-26T20:22:28.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.144+0000 7f6c7e693640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6c780b4400 con 0x7f6c78160d70 2024-04-26T20:22:28.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.144+0000 7f6c7e693640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6c78074570 con 0x7f6c7806f790 2024-04-26T20:22:28.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.144+0000 7f6c777fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c78160d70 0x7f6c78161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.144+0000 7f6c7cc30640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6c7815d780 0x7f6c7815dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57278/0 (socket says 172.21.15.37:57278) 2024-04-26T20:22:28.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.144+0000 7f6c7cc30640 1 -- 172.21.15.37:0/1346523472 learned_addr learned my addr 172.21.15.37:0/1346523472 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:28.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.145+0000 7f6c77fff640 1 --2- 172.21.15.37:0/1346523472 >> v2:172.21.15.134:3300/0 conn(0x7f6c7806f790 0x7f6c78074030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.145+0000 7f6c777fe640 1 -- 172.21.15.37:0/1346523472 >> v2:172.21.15.134:3300/0 conn(0x7f6c7806f790 msgr2=0x7f6c78074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.145+0000 7f6c777fe640 1 --2- 172.21.15.37:0/1346523472 >> v2:172.21.15.134:3300/0 conn(0x7f6c7806f790 0x7f6c78074030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.145+0000 7f6c777fe640 1 -- 172.21.15.37:0/1346523472 >> v1:172.21.15.37:6789/0 conn(0x7f6c7815d780 legacy=0x7f6c7815dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.145+0000 7f6c777fe640 1 -- 172.21.15.37:0/1346523472 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c780b4120 con 0x7f6c78160d70 2024-04-26T20:22:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.145+0000 7f6c76ffd640 1 -- 172.21.15.37:0/1346523472 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712349559 0 0) 0x7f6c78161b80 con 0x7f6c7815d780 2024-04-26T20:22:28.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.145+0000 7f6c777fe640 1 --2- 172.21.15.37:0/1346523472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c78160d70 0x7f6c78161200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6c78161d50 tx=0x7f6c6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca5e9cf2e9c2a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:28.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.146+0000 7f6c76ffd640 1 -- 172.21.15.37:0/1346523472 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68037070 con 0x7f6c78160d70 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.146+0000 7f6c76ffd640 1 -- 172.21.15.37:0/1346523472 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c68032440 con 0x7f6c78160d70 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.146+0000 7f6c76ffd640 1 -- 172.21.15.37:0/1346523472 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68032810 con 0x7f6c78160d70 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.146+0000 7f6c7e693640 1 -- 172.21.15.37:0/1346523472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c78160d70 msgr2=0x7f6c78161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.146+0000 7f6c7e693640 1 --2- 172.21.15.37:0/1346523472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c78160d70 0x7f6c78161200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6c78161d50 tx=0x7f6c6802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.146+0000 7f6c7e693640 1 -- 172.21.15.37:0/1346523472 shutdown_connections 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.146+0000 7f6c7e693640 1 --2- 172.21.15.37:0/1346523472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c78160d70 0x7f6c78161200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.146+0000 7f6c7e693640 1 --2- 172.21.15.37:0/1346523472 >> v2:172.21.15.134:3300/0 conn(0x7f6c7806f790 0x7f6c78074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.146+0000 7f6c7e693640 1 -- 172.21.15.37:0/1346523472 >> 172.21.15.37:0/1346523472 conn(0x7f6c780d0d50 msgr2=0x7f6c780d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.146+0000 7f6c7e693640 1 -- 172.21.15.37:0/1346523472 shutdown_connections 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.146+0000 7f6c7e693640 1 -- 172.21.15.37:0/1346523472 wait complete. 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.147+0000 7f6c7e693640 1 Processor -- start 2024-04-26T20:22:28.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.147+0000 7f6c7e693640 1 -- start start 2024-04-26T20:22:28.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.148+0000 7f6c7e693640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7806f790 0x7f6c780afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.148+0000 7f6c7e693640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7815d780 0x7f6c780ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.148+0000 7f6c7e693640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c78160d70 0x7f6c78143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.148+0000 7f6c7e693640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6c780735b0 con 0x7f6c78160d70 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.148+0000 7f6c7e693640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c780732d0 con 0x7f6c7806f790 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.148+0000 7f6c7e693640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c780b3d80 con 0x7f6c7815d780 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.148+0000 7f6c7cc30640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c78160d70 0x7f6c78143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.148+0000 7f6c7cc30640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c78160d70 0x7f6c78143f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57976/0 (socket says 172.21.15.37:57976) 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.148+0000 7f6c7cc30640 1 -- 172.21.15.37:0/842939221 learned_addr learned my addr 172.21.15.37:0/842939221 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.148+0000 7f6c77fff640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7806f790 0x7f6c780afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.148+0000 7f6c777fe640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7815d780 0x7f6c780ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c7cc30640 1 -- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7815d780 msgr2=0x7f6c780ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c7cc30640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7815d780 0x7f6c780ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c7cc30640 1 -- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7806f790 msgr2=0x7f6c780afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c7cc30640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7806f790 0x7f6c780afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c7cc30640 1 -- 172.21.15.37:0/842939221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c780b0100 con 0x7f6c78160d70 2024-04-26T20:22:28.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c777fe640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7815d780 0x7f6c780ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:28.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c7cc30640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c78160d70 0x7f6c78143f10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000c700 tx=0x7f6c7000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:28.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c74ff9640 1 -- 172.21.15.37:0/842939221 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70018070 con 0x7f6c78160d70 2024-04-26T20:22:28.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c74ff9640 1 -- 172.21.15.37:0/842939221 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c7000ce10 con 0x7f6c78160d70 2024-04-26T20:22:28.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c7e693640 1 -- 172.21.15.37:0/842939221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c781f91f0 con 0x7f6c78160d70 2024-04-26T20:22:28.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c74ff9640 1 -- 172.21.15.37:0/842939221 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000a860 con 0x7f6c78160d70 2024-04-26T20:22:28.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.149+0000 7f6c7e693640 1 -- 172.21.15.37:0/842939221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c781f97b0 con 0x7f6c78160d70 2024-04-26T20:22:28.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.150+0000 7f6c7e693640 1 -- 172.21.15.37:0/842939221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c78069c20 con 0x7f6c78160d70 2024-04-26T20:22:28.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.150+0000 7f6c74ff9640 1 -- 172.21.15.37:0/842939221 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c7001b070 con 0x7f6c78160d70 2024-04-26T20:22:28.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.151+0000 7f6c74ff9640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c5806c8f0 0x7f6c5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.151+0000 7f6c74ff9640 1 -- 172.21.15.37:0/842939221 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6c7008b010 con 0x7f6c78160d70 2024-04-26T20:22:28.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.151+0000 7f6c77fff640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c5806c8f0 0x7f6c5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.152+0000 7f6c77fff640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c5806c8f0 0x7f6c5806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000aa20 tx=0x7f6c60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:28.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.155+0000 7f6c74ff9640 1 -- 172.21.15.37:0/842939221 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6c78069c20 con 0x7f6c78160d70 2024-04-26T20:22:28.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.379+0000 7f6c7e693640 1 -- 172.21.15.37:0/842939221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c7806a360 con 0x7f6c78160d70 2024-04-26T20:22:28.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.379+0000 7f6c74ff9640 1 -- 172.21.15.37:0/842939221 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7f6c780b0100 con 0x7f6c78160d70 2024-04-26T20:22:28.380 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:28.380 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2911741012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2911741012},{"type":"v1","addr":"172.21.15.134:6836","nonce":2911741012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1600483467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1600483467},{"type":"v1","addr":"172.21.15.37:6835","nonce":1600483467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2722614558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2722614558},{"type":"v1","addr":"172.21.15.134:6837","nonce":2722614558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:21:49.237855+0000","modified":"2024-04-26T20:21:52.255148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3920647125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3920647125},{"type":"v1","addr":"172.21.15.37:6834","nonce":3920647125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-04-26T20:22:28.381 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-26T20:22:28.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 -- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c5806c8f0 msgr2=0x7f6c5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c5806c8f0 0x7f6c5806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000aa20 tx=0x7f6c60008040 comp rx=0 tx=0).stop 2024-04-26T20:22:28.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 -- 172.21.15.37:0/842939221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c78160d70 msgr2=0x7f6c78143f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c78160d70 0x7f6c78143f10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000c700 tx=0x7f6c7000cbc0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 -- 172.21.15.37:0/842939221 shutdown_connections 2024-04-26T20:22:28.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c78160d70 0x7f6c78143f10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c5806c8f0 0x7f6c5806edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7815d780 0x7f6c780ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 --2- 172.21.15.37:0/842939221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7806f790 0x7f6c780afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 -- 172.21.15.37:0/842939221 >> 172.21.15.37:0/842939221 conn(0x7f6c780d0d50 msgr2=0x7f6c78072540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:28.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 -- 172.21.15.37:0/842939221 shutdown_connections 2024-04-26T20:22:28.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.381+0000 7f6c7e693640 1 -- 172.21.15.37:0/842939221 wait complete. 2024-04-26T20:22:28.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:28.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.493+0000 7f2e5b8d1640 1 Processor -- start 2024-04-26T20:22:28.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.493+0000 7f2e5b8d1640 1 -- start start 2024-04-26T20:22:28.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.493+0000 7f2e5b8d1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2e5415f380 0x7f2e540d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e5b8d1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e540d3de0 con 0x7f2e5415f760 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e5b8d1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e540d40c0 con 0x7f2e540b9a90 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e5b8d1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2e540d43a0 con 0x7f2e5415f380 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e58e6c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2e5415f760 0x7f2e540cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57288/0 (socket says 172.21.15.37:57288) 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e58e6c640 1 -- 172.21.15.37:0/139442418 learned_addr learned my addr 172.21.15.37:0/139442418 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2016298567 0 0) 0x7f2e540d3de0 con 0x7f2e5415f760 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e5966d640 1 --2- 172.21.15.37:0/139442418 >> v2:172.21.15.134:3300/0 conn(0x7f2e5415f380 0x7f2e540d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e40003520 con 0x7f2e5415f760 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4282531740 0 0) 0x7f2e40003520 con 0x7f2e5415f760 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 >> v2:172.21.15.134:3300/0 conn(0x7f2e5415f380 msgr2=0x7f2e540d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 --2- 172.21.15.37:0/139442418 >> v2:172.21.15.134:3300/0 conn(0x7f2e5415f380 0x7f2e540d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 >> v1:172.21.15.134:6789/0 conn(0x7f2e540b9a90 legacy=0x7f2e540d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e540b8280 con 0x7f2e5415f760 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f2e44001670 con 0x7f2e5415f760 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 >> v1:172.21.15.37:6789/0 conn(0x7f2e5415f760 legacy=0x7f2e540cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 --2- 172.21.15.37:0/139442418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e400039b0 0x7f2e40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 --2- 172.21.15.37:0/139442418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e40004800 0x7f2e40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 --2- 172.21.15.37:0/139442418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e400051c0 0x7f2e4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e540d43a0 con 0x7f2e40004800 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4000c070 con 0x7f2e400039b0 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.494+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4000c270 con 0x7f2e400051c0 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.495+0000 7f2e58e6c640 1 --2- 172.21.15.37:0/139442418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e40004800 0x7f2e40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.495+0000 7f2e5966d640 1 --2- 172.21.15.37:0/139442418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e400039b0 0x7f2e40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.495+0000 7f2e59e6e640 1 --2- 172.21.15.37:0/139442418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e400051c0 0x7f2e4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.495+0000 7f2e58e6c640 1 -- 172.21.15.37:0/139442418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e400051c0 msgr2=0x7f2e4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.495+0000 7f2e58e6c640 1 --2- 172.21.15.37:0/139442418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e400051c0 0x7f2e4000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.495+0000 7f2e58e6c640 1 -- 172.21.15.37:0/139442418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e400039b0 msgr2=0x7f2e40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.495+0000 7f2e58e6c640 1 --2- 172.21.15.37:0/139442418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e400039b0 0x7f2e40003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.495+0000 7f2e58e6c640 1 -- 172.21.15.37:0/139442418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e40003520 con 0x7f2e40004800 2024-04-26T20:22:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.495+0000 7f2e59e6e640 1 -- 172.21.15.37:0/139442418 reap_dead start 2024-04-26T20:22:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.496+0000 7f2e58e6c640 1 --2- 172.21.15.37:0/139442418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e40004800 0x7f2e40004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400c210 tx=0x7f2e40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d6308e80c441e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.496+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44007cf0 con 0x7f2e40004800 2024-04-26T20:22:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.496+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e44005af0 con 0x7f2e40004800 2024-04-26T20:22:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.496+0000 7f2e4bfff640 1 -- 172.21.15.37:0/139442418 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44007070 con 0x7f2e40004800 2024-04-26T20:22:28.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.496+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/139442418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e40004800 msgr2=0x7f2e40004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.496+0000 7f2e5b8d1640 1 --2- 172.21.15.37:0/139442418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e40004800 0x7f2e40004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400c210 tx=0x7f2e40005830 comp rx=0 tx=0).stop 2024-04-26T20:22:28.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.496+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/139442418 shutdown_connections 2024-04-26T20:22:28.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.496+0000 7f2e5b8d1640 1 --2- 172.21.15.37:0/139442418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e40004800 0x7f2e40004c80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.496+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/139442418 >> 172.21.15.37:0/139442418 conn(0x7f2e5409e560 msgr2=0x7f2e540d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:28.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.496+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/139442418 shutdown_connections 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.496+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/139442418 wait complete. 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.498+0000 7f2e5b8d1640 1 Processor -- start 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.498+0000 7f2e5b8d1640 1 -- start start 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.498+0000 7f2e5b8d1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40004800 0x7f2e54151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.498+0000 7f2e5b8d1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e540c9880 0x7f2e540c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.498+0000 7f2e5b8d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e540cb660 0x7f2e540cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.498+0000 7f2e5b8d1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e540d1280 con 0x7f2e540cb660 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.498+0000 7f2e5b8d1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e540d1100 con 0x7f2e40004800 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.498+0000 7f2e5b8d1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2e540d5b50 con 0x7f2e540c9880 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.498+0000 7f2e59e6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e540cb660 0x7f2e540cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.498+0000 7f2e59e6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e540cb660 0x7f2e540cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57986/0 (socket says 172.21.15.37:57986) 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.498+0000 7f2e59e6e640 1 -- 172.21.15.37:0/3440419121 learned_addr learned my addr 172.21.15.37:0/3440419121 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e58e6c640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e540c9880 0x7f2e540c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e5966d640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40004800 0x7f2e54151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e59e6e640 1 -- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e540c9880 msgr2=0x7f2e540c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e59e6e640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e540c9880 0x7f2e540c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e59e6e640 1 -- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40004800 msgr2=0x7f2e54151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e59e6e640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40004800 0x7f2e54151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e59e6e640 1 -- 172.21.15.37:0/3440419121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e540ca2d0 con 0x7f2e540cb660 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e59e6e640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e540cb660 0x7f2e540cbae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00f7e0 tx=0x7f2e4c00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e49ffb640 1 -- 172.21.15.37:0/3440419121 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c003970 con 0x7f2e540cb660 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e49ffb640 1 -- 172.21.15.37:0/3440419121 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4c018070 con 0x7f2e540cb660 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/3440419121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e540ca5b0 con 0x7f2e540cb660 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e49ffb640 1 -- 172.21.15.37:0/3440419121 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c003d70 con 0x7f2e540cb660 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.499+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/3440419121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e540cab70 con 0x7f2e540cb660 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.500+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/3440419121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e5406a360 con 0x7f2e540cb660 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.501+0000 7f2e49ffb640 1 -- 172.21.15.37:0/3440419121 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e4c0050d0 con 0x7f2e540cb660 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.501+0000 7f2e49ffb640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2e3806c8f0 0x7f2e3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.501+0000 7f2e49ffb640 1 -- 172.21.15.37:0/3440419121 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2e4c08b7e0 con 0x7f2e540cb660 2024-04-26T20:22:28.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.501+0000 7f2e5966d640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2e3806c8f0 0x7f2e3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.502+0000 7f2e5966d640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2e3806c8f0 0x7f2e3806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009bc0 tx=0x7f2e50009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:28.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.505+0000 7f2e49ffb640 1 -- 172.21.15.37:0/3440419121 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e4c059c10 con 0x7f2e540cb660 2024-04-26T20:22:28.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.691+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/3440419121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e540d24e0 con 0x7f2e540cb660 2024-04-26T20:22:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.692+0000 7f2e49ffb640 1 -- 172.21.15.37:0/3440419121 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f2e4c059510 con 0x7f2e540cb660 2024-04-26T20:22:28.693 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:28.694 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:21:50.880684+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:28.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.694+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2e3806c8f0 msgr2=0x7f2e3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.694+0000 7f2e5b8d1640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2e3806c8f0 0x7f2e3806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009bc0 tx=0x7f2e50009190 comp rx=0 tx=0).stop 2024-04-26T20:22:28.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.694+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/3440419121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e540cb660 msgr2=0x7f2e540cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.694+0000 7f2e5b8d1640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e540cb660 0x7f2e540cbae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00f7e0 tx=0x7f2e4c00fbe0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.694+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/3440419121 shutdown_connections 2024-04-26T20:22:28.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.695+0000 7f2e5b8d1640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e540cb660 0x7f2e540cbae0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.695+0000 7f2e5b8d1640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2e3806c8f0 0x7f2e3806edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.695+0000 7f2e5b8d1640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e540c9880 0x7f2e540c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.695+0000 7f2e5b8d1640 1 --2- 172.21.15.37:0/3440419121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40004800 0x7f2e54151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.695+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/3440419121 >> 172.21.15.37:0/3440419121 conn(0x7f2e5409e560 msgr2=0x7f2e5406ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:28.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.695+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/3440419121 shutdown_connections 2024-04-26T20:22:28.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.695+0000 7f2e5b8d1640 1 -- 172.21.15.37:0/3440419121 wait complete. 2024-04-26T20:22:28.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:21:47.476747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:21:48.186397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:28.707 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:28.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.810+0000 7fcf896d3640 1 Processor -- start 2024-04-26T20:22:28.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.810+0000 7fcf896d3640 1 -- start start 2024-04-26T20:22:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf896d3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf840d5a80 0x7fcf840b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf896d3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcf840b4b20 0x7fcf841630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf896d3640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf840b2760 con 0x7fcf8415f400 2024-04-26T20:22:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf896d3640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcf840b43e0 con 0x7fcf840b4b20 2024-04-26T20:22:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf896d3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcf840b3ed0 con 0x7fcf840d5a80 2024-04-26T20:22:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf827fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf840d5a80 0x7fcf840b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf837fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcf840b4b20 0x7fcf841630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf827fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf840d5a80 0x7fcf840b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:53728/0 (socket says 172.21.15.37:53728) 2024-04-26T20:22:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf827fc640 1 -- 172.21.15.37:0/1772852697 learned_addr learned my addr 172.21.15.37:0/1772852697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf837fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcf840b4b20 0x7fcf841630b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:55606/0 (socket says 172.21.15.37:55606) 2024-04-26T20:22:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf827fc640 1 -- 172.21.15.37:0/1772852697 >> v2:172.21.15.134:3300/0 conn(0x7fcf840b4b20 msgr2=0x7fcf841630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf827fc640 1 --2- 172.21.15.37:0/1772852697 >> v2:172.21.15.134:3300/0 conn(0x7fcf840b4b20 0x7fcf841630b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf827fc640 1 -- 172.21.15.37:0/1772852697 >> v1:172.21.15.134:6789/0 conn(0x7fcf8415f400 legacy=0x7fcf840b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf827fc640 1 -- 172.21.15.37:0/1772852697 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf840b2a00 con 0x7fcf840d5a80 2024-04-26T20:22:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.811+0000 7fcf837fe640 1 --2- 172.21.15.37:0/1772852697 >> v2:172.21.15.134:3300/0 conn(0x7fcf840b4b20 0x7fcf841630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.812+0000 7fcf827fc640 1 --2- 172.21.15.37:0/1772852697 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf840d5a80 0x7fcf840b45e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009870 tx=0x7fcf7802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f77752079aac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.812+0000 7fcf81ffb640 1 -- 172.21.15.37:0/1772852697 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78037070 con 0x7fcf840d5a80 2024-04-26T20:22:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.812+0000 7fcf81ffb640 1 -- 172.21.15.37:0/1772852697 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf78032440 con 0x7fcf840d5a80 2024-04-26T20:22:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.812+0000 7fcf81ffb640 1 -- 172.21.15.37:0/1772852697 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78032810 con 0x7fcf840d5a80 2024-04-26T20:22:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.813+0000 7fcf896d3640 1 -- 172.21.15.37:0/1772852697 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf840d5a80 msgr2=0x7fcf840b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.813+0000 7fcf896d3640 1 --2- 172.21.15.37:0/1772852697 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf840d5a80 0x7fcf840b45e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009870 tx=0x7fcf7802a250 comp rx=0 tx=0).stop 2024-04-26T20:22:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.813+0000 7fcf896d3640 1 -- 172.21.15.37:0/1772852697 shutdown_connections 2024-04-26T20:22:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.813+0000 7fcf896d3640 1 --2- 172.21.15.37:0/1772852697 >> v2:172.21.15.134:3300/0 conn(0x7fcf840b4b20 0x7fcf841630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.813+0000 7fcf896d3640 1 --2- 172.21.15.37:0/1772852697 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf840d5a80 0x7fcf840b45e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.813+0000 7fcf896d3640 1 -- 172.21.15.37:0/1772852697 >> 172.21.15.37:0/1772852697 conn(0x7fcf8409e4f0 msgr2=0x7fcf840d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:28.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.813+0000 7fcf896d3640 1 -- 172.21.15.37:0/1772852697 shutdown_connections 2024-04-26T20:22:28.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.813+0000 7fcf896d3640 1 -- 172.21.15.37:0/1772852697 wait complete. 2024-04-26T20:22:28.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.814+0000 7fcf896d3640 1 Processor -- start 2024-04-26T20:22:28.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.814+0000 7fcf896d3640 1 -- start start 2024-04-26T20:22:28.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf896d3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf840b4b20 0x7fcf840cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf896d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf840d5a80 0x7fcf840d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf896d3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf8415f400 0x7fcf840c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf896d3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf840b32a0 con 0x7fcf840d5a80 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf896d3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf840b2fc0 con 0x7fcf840b4b20 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf896d3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcf840b4f70 con 0x7fcf8415f400 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf827fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf840d5a80 0x7fcf840d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf827fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf840d5a80 0x7fcf840d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57994/0 (socket says 172.21.15.37:57994) 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf827fc640 1 -- 172.21.15.37:0/1594601328 learned_addr learned my addr 172.21.15.37:0/1594601328 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf837fe640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf8415f400 0x7fcf840c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf82ffd640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf840b4b20 0x7fcf840cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf827fc640 1 -- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf8415f400 msgr2=0x7fcf840c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf827fc640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf8415f400 0x7fcf840c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf827fc640 1 -- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf840b4b20 msgr2=0x7fcf840cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf827fc640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf840b4b20 0x7fcf840cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.815+0000 7fcf827fc640 1 -- 172.21.15.37:0/1594601328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf840ca3f0 con 0x7fcf840d5a80 2024-04-26T20:22:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.816+0000 7fcf827fc640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf840d5a80 0x7fcf840d0a60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcf7802a290 tx=0x7fcf780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.816+0000 7fcf6b7fe640 1 -- 172.21.15.37:0/1594601328 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78037040 con 0x7fcf840d5a80 2024-04-26T20:22:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.816+0000 7fcf6b7fe640 1 -- 172.21.15.37:0/1594601328 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf78042070 con 0x7fcf840d5a80 2024-04-26T20:22:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.816+0000 7fcf896d3640 1 -- 172.21.15.37:0/1594601328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf840ca6d0 con 0x7fcf840d5a80 2024-04-26T20:22:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.816+0000 7fcf6b7fe640 1 -- 172.21.15.37:0/1594601328 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78030880 con 0x7fcf840d5a80 2024-04-26T20:22:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.816+0000 7fcf896d3640 1 -- 172.21.15.37:0/1594601328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf840cac90 con 0x7fcf840d5a80 2024-04-26T20:22:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.817+0000 7fcf896d3640 1 -- 172.21.15.37:0/1594601328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf840cb470 con 0x7fcf840d5a80 2024-04-26T20:22:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.817+0000 7fcf6b7fe640 1 -- 172.21.15.37:0/1594601328 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf78030a20 con 0x7fcf840d5a80 2024-04-26T20:22:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.818+0000 7fcf6b7fe640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcf7006c8f0 0x7fcf7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.818+0000 7fcf6b7fe640 1 -- 172.21.15.37:0/1594601328 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf780ac350 con 0x7fcf840d5a80 2024-04-26T20:22:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.818+0000 7fcf82ffd640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcf7006c8f0 0x7fcf7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.819+0000 7fcf82ffd640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcf7006c8f0 0x7fcf7006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcf74001640 tx=0x7fcf74009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:28.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:28.822+0000 7fcf6b7fe640 1 -- 172.21.15.37:0/1594601328 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf840cb470 con 0x7fcf840d5a80 2024-04-26T20:22:29.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.008+0000 7fcf896d3640 1 -- 172.21.15.37:0/1594601328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf8406a360 con 0x7fcf840d5a80 2024-04-26T20:22:29.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.008+0000 7fcf6b7fe640 1 -- 172.21.15.37:0/1594601328 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fcf78045360 con 0x7fcf840d5a80 2024-04-26T20:22:29.009 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:29.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 -- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcf7006c8f0 msgr2=0x7fcf7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcf7006c8f0 0x7fcf7006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcf74001640 tx=0x7fcf74009040 comp rx=0 tx=0).stop 2024-04-26T20:22:29.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 -- 172.21.15.37:0/1594601328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf840d5a80 msgr2=0x7fcf840d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf840d5a80 0x7fcf840d0a60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcf7802a290 tx=0x7fcf780046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 -- 172.21.15.37:0/1594601328 shutdown_connections 2024-04-26T20:22:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcf7006c8f0 0x7fcf7006edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf8415f400 0x7fcf840c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf840d5a80 0x7fcf840d0a60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 --2- 172.21.15.37:0/1594601328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf840b4b20 0x7fcf840cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 -- 172.21.15.37:0/1594601328 >> 172.21.15.37:0/1594601328 conn(0x7fcf8409e4f0 msgr2=0x7fcf8406e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 -- 172.21.15.37:0/1594601328 shutdown_connections 2024-04-26T20:22:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.010+0000 7fcf896d3640 1 -- 172.21.15.37:0/1594601328 wait complete. 2024-04-26T20:22:29.023 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:22:29.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.125+0000 7fdc73340640 1 Processor -- start 2024-04-26T20:22:29.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.125+0000 7fdc73340640 1 -- start start 2024-04-26T20:22:29.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.126+0000 7fdc73340640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdc6c0d5a80 0x7fdc6c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.126+0000 7fdc73340640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc6c0b2760 con 0x7fdc6c15f400 2024-04-26T20:22:29.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.126+0000 7fdc73340640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc6c0b2a00 con 0x7fdc6c0b4b20 2024-04-26T20:22:29.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.126+0000 7fdc73340640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fdc6c0b43e0 con 0x7fdc6c0d5a80 2024-04-26T20:22:29.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.126+0000 7fdc710dc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdc6c15f400 0x7fdc6c0b37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57300/0 (socket says 172.21.15.37:57300) 2024-04-26T20:22:29.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.126+0000 7fdc710dc640 1 -- 172.21.15.37:0/2992218234 learned_addr learned my addr 172.21.15.37:0/2992218234 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:29.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.126+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3735744114 0 0) 0x7fdc6c0b2760 con 0x7fdc6c15f400 2024-04-26T20:22:29.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.126+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc54003520 con 0x7fdc6c15f400 2024-04-26T20:22:29.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.126+0000 7fdc708db640 1 --2- 172.21.15.37:0/2992218234 >> v2:172.21.15.134:3300/0 conn(0x7fdc6c0d5a80 0x7fdc6c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 386230016 0 0) 0x7fdc54003520 con 0x7fdc6c15f400 2024-04-26T20:22:29.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 >> v2:172.21.15.134:3300/0 conn(0x7fdc6c0d5a80 msgr2=0x7fdc6c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 --2- 172.21.15.37:0/2992218234 >> v2:172.21.15.134:3300/0 conn(0x7fdc6c0d5a80 0x7fdc6c0b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 >> v1:172.21.15.134:6789/0 conn(0x7fdc6c0b4b20 legacy=0x7fdc6c1630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c0b2ce0 con 0x7fdc6c15f400 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fdc64001690 con 0x7fdc6c15f400 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 >> v1:172.21.15.37:6789/0 conn(0x7fdc6c15f400 legacy=0x7fdc6c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 --2- 172.21.15.37:0/2992218234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc540039b0 0x7fdc54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 --2- 172.21.15.37:0/2992218234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 --2- 172.21.15.37:0/2992218234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc540052e0 0x7fdc5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5400c180 con 0x7fdc54004800 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5400c330 con 0x7fdc540039b0 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5400c530 con 0x7fdc540052e0 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc708db640 1 --2- 172.21.15.37:0/2992218234 >> v2:172.21.15.134:3300/0 conn(0x7fdc6c0d5a80 0x7fdc6c0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc708db640 1 --2- 172.21.15.37:0/2992218234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc710dc640 1 --2- 172.21.15.37:0/2992218234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc540039b0 0x7fdc54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc718dd640 1 --2- 172.21.15.37:0/2992218234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc540052e0 0x7fdc5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc708db640 1 -- 172.21.15.37:0/2992218234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc540052e0 msgr2=0x7fdc5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc708db640 1 --2- 172.21.15.37:0/2992218234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc540052e0 0x7fdc5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc708db640 1 -- 172.21.15.37:0/2992218234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc540039b0 msgr2=0x7fdc54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc708db640 1 --2- 172.21.15.37:0/2992218234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc540039b0 0x7fdc54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc708db640 1 -- 172.21.15.37:0/2992218234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc54003520 con 0x7fdc54004800 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.127+0000 7fdc718dd640 1 -- 172.21.15.37:0/2992218234 reap_dead start 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.128+0000 7fdc708db640 1 --2- 172.21.15.37:0/2992218234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdc60006fd0 tx=0x7fdc54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b2980ee971624f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.128+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60037070 con 0x7fdc54004800 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.128+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6002ac50 con 0x7fdc54004800 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.128+0000 7fdc6bfff640 1 -- 172.21.15.37:0/2992218234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000f040 con 0x7fdc54004800 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.129+0000 7fdc73340640 1 -- 172.21.15.37:0/2992218234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc54004800 msgr2=0x7fdc54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.129+0000 7fdc73340640 1 --2- 172.21.15.37:0/2992218234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdc60006fd0 tx=0x7fdc54005830 comp rx=0 tx=0).stop 2024-04-26T20:22:29.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.129+0000 7fdc73340640 1 -- 172.21.15.37:0/2992218234 shutdown_connections 2024-04-26T20:22:29.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.129+0000 7fdc73340640 1 --2- 172.21.15.37:0/2992218234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.129+0000 7fdc73340640 1 -- 172.21.15.37:0/2992218234 >> 172.21.15.37:0/2992218234 conn(0x7fdc6c09e4f0 msgr2=0x7fdc6c0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:29.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.129+0000 7fdc73340640 1 -- 172.21.15.37:0/2992218234 shutdown_connections 2024-04-26T20:22:29.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.129+0000 7fdc73340640 1 -- 172.21.15.37:0/2992218234 wait complete. 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.130+0000 7fdc73340640 1 Processor -- start 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.130+0000 7fdc73340640 1 -- start start 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc73340640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc54004800 0x7fdc6c1f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc73340640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1f5210 0x7fdc6c1f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc73340640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc6c0b8090 0x7fdc6c0b8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc73340640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c0b4f30 con 0x7fdc6c1f5210 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc73340640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c0b32a0 con 0x7fdc6c0b8090 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc73340640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6c0b50b0 con 0x7fdc54004800 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc708db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1f5210 0x7fdc6c1f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc708db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1f5210 0x7fdc6c1f8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58008/0 (socket says 172.21.15.37:58008) 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc708db640 1 -- 172.21.15.37:0/2579017945 learned_addr learned my addr 172.21.15.37:0/2579017945 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc718dd640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc6c0b8090 0x7fdc6c0b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc710dc640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc54004800 0x7fdc6c1f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc708db640 1 -- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc54004800 msgr2=0x7fdc6c1f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc708db640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc54004800 0x7fdc6c1f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc708db640 1 -- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc6c0b8090 msgr2=0x7fdc6c0b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc708db640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc6c0b8090 0x7fdc6c0b8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc708db640 1 -- 172.21.15.37:0/2579017945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c1f95a0 con 0x7fdc6c1f5210 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.131+0000 7fdc708db640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1f5210 0x7fdc6c1f8690 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdc6002a740 tx=0x7fdc60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.132+0000 7fdc69ffb640 1 -- 172.21.15.37:0/2579017945 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60040020 con 0x7fdc6c1f5210 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.132+0000 7fdc69ffb640 1 -- 172.21.15.37:0/2579017945 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc60038040 con 0x7fdc6c1f5210 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.132+0000 7fdc73340640 1 -- 172.21.15.37:0/2579017945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6c1f8fe0 con 0x7fdc6c1f5210 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.132+0000 7fdc69ffb640 1 -- 172.21.15.37:0/2579017945 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60037070 con 0x7fdc6c1f5210 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.132+0000 7fdc73340640 1 -- 172.21.15.37:0/2579017945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc6c0cd320 con 0x7fdc6c1f5210 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.133+0000 7fdc73340640 1 -- 172.21.15.37:0/2579017945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc6c06a360 con 0x7fdc6c1f5210 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.133+0000 7fdc69ffb640 1 -- 172.21.15.37:0/2579017945 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc60043020 con 0x7fdc6c1f5210 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.134+0000 7fdc69ffb640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdc5806c8f0 0x7fdc5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.134+0000 7fdc69ffb640 1 -- 172.21.15.37:0/2579017945 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdc600abc10 con 0x7fdc6c1f5210 2024-04-26T20:22:29.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.134+0000 7fdc710dc640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdc5806c8f0 0x7fdc5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.134+0000 7fdc710dc640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdc5806c8f0 0x7fdc5806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdc64001200 tx=0x7fdc64001110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.138+0000 7fdc69ffb640 1 -- 172.21.15.37:0/2579017945 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc6004a020 con 0x7fdc6c1f5210 2024-04-26T20:22:29.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.323+0000 7fdc73340640 1 -- 172.21.15.37:0/2579017945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdc6c0d28f0 con 0x7fdc6c1f5210 2024-04-26T20:22:29.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.323+0000 7fdc69ffb640 1 -- 172.21.15.37:0/2579017945 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fdc6000f000 con 0x7fdc6c1f5210 2024-04-26T20:22:29.324 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:22:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 -- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdc5806c8f0 msgr2=0x7fdc5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdc5806c8f0 0x7fdc5806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdc64001200 tx=0x7fdc64001110 comp rx=0 tx=0).stop 2024-04-26T20:22:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 -- 172.21.15.37:0/2579017945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1f5210 msgr2=0x7fdc6c1f8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1f5210 0x7fdc6c1f8690 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdc6002a740 tx=0x7fdc60004730 comp rx=0 tx=0).stop 2024-04-26T20:22:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 -- 172.21.15.37:0/2579017945 shutdown_connections 2024-04-26T20:22:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc6c0b8090 0x7fdc6c0b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1f5210 0x7fdc6c1f8690 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdc5806c8f0 0x7fdc5806edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 --2- 172.21.15.37:0/2579017945 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc54004800 0x7fdc6c1f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 -- 172.21.15.37:0/2579017945 >> 172.21.15.37:0/2579017945 conn(0x7fdc6c09e4f0 msgr2=0x7fdc6c15e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 -- 172.21.15.37:0/2579017945 shutdown_connections 2024-04-26T20:22:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.325+0000 7fdc73340640 1 -- 172.21.15.37:0/2579017945 wait complete. 2024-04-26T20:22:29.337 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:22:29.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.440+0000 7f0d324ea640 1 Processor -- start 2024-04-26T20:22:29.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.440+0000 7f0d324ea640 1 -- start start 2024-04-26T20:22:29.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.440+0000 7f0d324ea640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0d2c0d2320 0x7f0d2c0d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.440+0000 7f0d324ea640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d2c161510 con 0x7f0d2c1610c0 2024-04-26T20:22:29.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.440+0000 7f0d324ea640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d2c0d0930 con 0x7f0d2c160800 2024-04-26T20:22:29.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.440+0000 7f0d324ea640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f0d2c0d0c00 con 0x7f0d2c0d2320 2024-04-26T20:22:29.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.440+0000 7f0d2b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0d2c1610c0 0x7f0d2c0d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57316/0 (socket says 172.21.15.37:57316) 2024-04-26T20:22:29.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.440+0000 7f0d2b7fe640 1 -- 172.21.15.37:0/2093934977 learned_addr learned my addr 172.21.15.37:0/2093934977 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:29.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d30a87640 1 --2- 172.21.15.37:0/2093934977 >> v2:172.21.15.134:3300/0 conn(0x7f0d2c0d2320 0x7f0d2c0d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678313693 0 0) 0x7f0d2c161510 con 0x7f0d2c1610c0 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d18003520 con 0x7f0d2c1610c0 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2870780965 0 0) 0x7f0d18003520 con 0x7f0d2c1610c0 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 >> v2:172.21.15.134:3300/0 conn(0x7f0d2c0d2320 msgr2=0x7f0d2c0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 --2- 172.21.15.37:0/2093934977 >> v2:172.21.15.134:3300/0 conn(0x7f0d2c0d2320 0x7f0d2c0d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 >> v1:172.21.15.134:6789/0 conn(0x7f0d2c160800 legacy=0x7f0d2c0d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2c0d0ee0 con 0x7f0d2c1610c0 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f0d1c001670 con 0x7f0d2c1610c0 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 >> v1:172.21.15.37:6789/0 conn(0x7f0d2c1610c0 legacy=0x7f0d2c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d180039b0 0x7f0d18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d18004800 0x7f0d18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d180051c0 0x7f0d1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c0d0c00 con 0x7f0d180051c0 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1800c160 con 0x7f0d18004800 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1800c360 con 0x7f0d180039b0 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d30a87640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d180051c0 0x7f0d1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2bfff640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d180039b0 0x7f0d18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d2b7fe640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d18004800 0x7f0d18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d30a87640 1 -- 172.21.15.37:0/2093934977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d180039b0 msgr2=0x7f0d18003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d30a87640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d180039b0 0x7f0d18003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d30a87640 1 -- 172.21.15.37:0/2093934977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d18004800 msgr2=0x7f0d18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d30a87640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d18004800 0x7f0d18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.441+0000 7f0d30a87640 1 -- 172.21.15.37:0/2093934977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d18003520 con 0x7f0d180051c0 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.442+0000 7f0d2b7fe640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d18004800 0x7f0d18004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.442+0000 7f0d30a87640 1 -- 172.21.15.37:0/2093934977 reap_dead start 2024-04-26T20:22:29.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.442+0000 7f0d30a87640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d180051c0 0x7f0d1800b9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009fc0 tx=0x7f0d18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc27bc9cf9044eb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:29.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.442+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24035020 con 0x7f0d180051c0 2024-04-26T20:22:29.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.442+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d24003bd0 con 0x7f0d180051c0 2024-04-26T20:22:29.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.442+0000 7f0d2affd640 1 -- 172.21.15.37:0/2093934977 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24009c10 con 0x7f0d180051c0 2024-04-26T20:22:29.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.443+0000 7f0d324ea640 1 -- 172.21.15.37:0/2093934977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d180051c0 msgr2=0x7f0d1800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.443+0000 7f0d324ea640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d180051c0 0x7f0d1800b9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009fc0 tx=0x7f0d18005830 comp rx=0 tx=0).stop 2024-04-26T20:22:29.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.443+0000 7f0d324ea640 1 -- 172.21.15.37:0/2093934977 shutdown_connections 2024-04-26T20:22:29.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.443+0000 7f0d324ea640 1 --2- 172.21.15.37:0/2093934977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d180051c0 0x7f0d1800b9c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.443+0000 7f0d324ea640 1 -- 172.21.15.37:0/2093934977 >> 172.21.15.37:0/2093934977 conn(0x7f0d2c09e550 msgr2=0x7f0d2c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:29.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.443+0000 7f0d324ea640 1 -- 172.21.15.37:0/2093934977 shutdown_connections 2024-04-26T20:22:29.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.443+0000 7f0d324ea640 1 -- 172.21.15.37:0/2093934977 wait complete. 2024-04-26T20:22:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.444+0000 7f0d324ea640 1 Processor -- start 2024-04-26T20:22:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.444+0000 7f0d324ea640 1 -- start start 2024-04-26T20:22:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d324ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d180051c0 0x7f0d2c152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d324ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d2c1f5250 0x7f0d2c0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d324ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c0d14b0 0x7f0d2c0d1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d324ea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c0d30f0 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d324ea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c0d2e10 con 0x7f0d2c1f5250 2024-04-26T20:22:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d324ea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2c0b6300 con 0x7f0d180051c0 2024-04-26T20:22:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d30a87640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c0d14b0 0x7f0d2c0d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d30a87640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c0d14b0 0x7f0d2c0d1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58016/0 (socket says 172.21.15.37:58016) 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d30a87640 1 -- 172.21.15.37:0/336219676 learned_addr learned my addr 172.21.15.37:0/336219676 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d2b7fe640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d2c1f5250 0x7f0d2c0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d2bfff640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d180051c0 0x7f0d2c152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d30a87640 1 -- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d180051c0 msgr2=0x7f0d2c152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.445+0000 7f0d30a87640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d180051c0 0x7f0d2c152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.446+0000 7f0d30a87640 1 -- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d2c1f5250 msgr2=0x7f0d2c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.446+0000 7f0d30a87640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d2c1f5250 0x7f0d2c0ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.446+0000 7f0d30a87640 1 -- 172.21.15.37:0/336219676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2c1f5820 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.446+0000 7f0d2b7fe640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d2c1f5250 0x7f0d2c0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.446+0000 7f0d30a87640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c0d14b0 0x7f0d2c0d1930 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d2402af10 tx=0x7f0d24009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.446+0000 7f0d28ff9640 1 -- 172.21.15.37:0/336219676 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24035030 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.446+0000 7f0d28ff9640 1 -- 172.21.15.37:0/336219676 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d24043070 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.446+0000 7f0d324ea640 1 -- 172.21.15.37:0/336219676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2c1f5b00 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.446+0000 7f0d28ff9640 1 -- 172.21.15.37:0/336219676 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2402d070 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.446+0000 7f0d324ea640 1 -- 172.21.15.37:0/336219676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2c1f60c0 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.447+0000 7f0d324ea640 1 -- 172.21.15.37:0/336219676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d2c06a360 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.447+0000 7f0d28ff9640 1 -- 172.21.15.37:0/336219676 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d24005d10 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.448+0000 7f0d28ff9640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0d0c06c920 0x7f0d0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.448+0000 7f0d28ff9640 1 -- 172.21.15.37:0/336219676 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0d240ac810 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.448+0000 7f0d2bfff640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0d0c06c920 0x7f0d0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.448+0000 7f0d2bfff640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0d0c06c920 0x7f0d0c06ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0d14000c00 tx=0x7f0d14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.452+0000 7f0d28ff9640 1 -- 172.21.15.37:0/336219676 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0d2407ac40 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.637+0000 7f0d324ea640 1 -- 172.21.15.37:0/336219676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0d2c0d4530 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.638+0000 7f0d28ff9640 1 -- 172.21.15.37:0/336219676 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f0d2407a540 con 0x7f0d2c0d14b0 2024-04-26T20:22:29.639 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:22:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.639+0000 7f0d324ea640 1 -- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0d0c06c920 msgr2=0x7f0d0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.639+0000 7f0d324ea640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0d0c06c920 0x7f0d0c06ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0d14000c00 tx=0x7f0d14009040 comp rx=0 tx=0).stop 2024-04-26T20:22:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.639+0000 7f0d324ea640 1 -- 172.21.15.37:0/336219676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c0d14b0 msgr2=0x7f0d2c0d1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.639+0000 7f0d324ea640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c0d14b0 0x7f0d2c0d1930 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d2402af10 tx=0x7f0d24009750 comp rx=0 tx=0).stop 2024-04-26T20:22:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.639+0000 7f0d324ea640 1 -- 172.21.15.37:0/336219676 shutdown_connections 2024-04-26T20:22:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.639+0000 7f0d324ea640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c0d14b0 0x7f0d2c0d1930 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.639+0000 7f0d324ea640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d2c1f5250 0x7f0d2c0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.639+0000 7f0d324ea640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0d0c06c920 0x7f0d0c06ede0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.639+0000 7f0d324ea640 1 --2- 172.21.15.37:0/336219676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d180051c0 0x7f0d2c152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.639+0000 7f0d324ea640 1 -- 172.21.15.37:0/336219676 >> 172.21.15.37:0/336219676 conn(0x7f0d2c09e550 msgr2=0x7f0d2c0cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.640+0000 7f0d324ea640 1 -- 172.21.15.37:0/336219676 shutdown_connections 2024-04-26T20:22:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.640+0000 7f0d324ea640 1 -- 172.21.15.37:0/336219676 wait complete. 2024-04-26T20:22:29.651 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:22:29.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.750+0000 7fb8242fc640 1 Processor -- start 2024-04-26T20:22:29.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.750+0000 7fb8242fc640 1 -- start start 2024-04-26T20:22:29.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.751+0000 7fb8242fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb81c15f380 0x7fb81c0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.751+0000 7fb8242fc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb81c15f760 0x7fb81c0d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.751+0000 7fb8242fc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb81c0d3de0 con 0x7fb81c0d5b50 2024-04-26T20:22:29.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.751+0000 7fb8242fc640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb81c0d43a0 con 0x7fb81c15f760 2024-04-26T20:22:29.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.751+0000 7fb8242fc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb81c0d40c0 con 0x7fb81c15f380 2024-04-26T20:22:29.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.751+0000 7fb822899640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb81c0d5b50 0x7fb81c0cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57326/0 (socket says 172.21.15.37:57326) 2024-04-26T20:22:29.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.751+0000 7fb822899640 1 -- 172.21.15.37:0/781025054 learned_addr learned my addr 172.21.15.37:0/781025054 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:29.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.751+0000 7fb821897640 1 --2- 172.21.15.37:0/781025054 >> v2:172.21.15.134:3300/0 conn(0x7fb81c15f760 0x7fb81c0d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.751+0000 7fb822098640 1 --2- 172.21.15.37:0/781025054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb81c15f380 0x7fb81c0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.752+0000 7fb821096640 1 -- 172.21.15.37:0/781025054 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 904619526 0 0) 0x7fb81c0d3de0 con 0x7fb81c0d5b50 2024-04-26T20:22:29.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.752+0000 7fb821096640 1 -- 172.21.15.37:0/781025054 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb804003520 con 0x7fb81c0d5b50 2024-04-26T20:22:29.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.752+0000 7fb822098640 1 -- 172.21.15.37:0/781025054 >> v2:172.21.15.134:3300/0 conn(0x7fb81c15f760 msgr2=0x7fb81c0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:29.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.752+0000 7fb822098640 1 --2- 172.21.15.37:0/781025054 >> v2:172.21.15.134:3300/0 conn(0x7fb81c15f760 0x7fb81c0d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.752+0000 7fb822098640 1 -- 172.21.15.37:0/781025054 >> v1:172.21.15.37:6789/0 conn(0x7fb81c0d5b50 legacy=0x7fb81c0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.752+0000 7fb822098640 1 -- 172.21.15.37:0/781025054 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c0b8280 con 0x7fb81c15f380 2024-04-26T20:22:29.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.752+0000 7fb821897640 1 --2- 172.21.15.37:0/781025054 >> v2:172.21.15.134:3300/0 conn(0x7fb81c15f760 0x7fb81c0d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:29.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb822098640 1 --2- 172.21.15.37:0/781025054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb81c15f380 0x7fb81c0d1bd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb8140097a0 tx=0x7fb81402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f395fcaca9ab1b5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb821096640 1 -- 172.21.15.37:0/781025054 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814037070 con 0x7fb81c15f380 2024-04-26T20:22:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb821096640 1 -- 172.21.15.37:0/781025054 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb814032440 con 0x7fb81c15f380 2024-04-26T20:22:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb821096640 1 -- 172.21.15.37:0/781025054 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814032810 con 0x7fb81c15f380 2024-04-26T20:22:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb8242fc640 1 -- 172.21.15.37:0/781025054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb81c15f380 msgr2=0x7fb81c0d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb8242fc640 1 --2- 172.21.15.37:0/781025054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb81c15f380 0x7fb81c0d1bd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb8140097a0 tx=0x7fb81402a210 comp rx=0 tx=0).stop 2024-04-26T20:22:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb8242fc640 1 -- 172.21.15.37:0/781025054 shutdown_connections 2024-04-26T20:22:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb8242fc640 1 --2- 172.21.15.37:0/781025054 >> v2:172.21.15.134:3300/0 conn(0x7fb81c15f760 0x7fb81c0d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb8242fc640 1 --2- 172.21.15.37:0/781025054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb81c15f380 0x7fb81c0d1bd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb8242fc640 1 -- 172.21.15.37:0/781025054 >> 172.21.15.37:0/781025054 conn(0x7fb81c09e560 msgr2=0x7fb81c0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb8242fc640 1 -- 172.21.15.37:0/781025054 shutdown_connections 2024-04-26T20:22:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.753+0000 7fb8242fc640 1 -- 172.21.15.37:0/781025054 wait complete. 2024-04-26T20:22:29.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.754+0000 7fb8242fc640 1 Processor -- start 2024-04-26T20:22:29.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.755+0000 7fb8242fc640 1 -- start start 2024-04-26T20:22:29.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.755+0000 7fb8242fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb81c0d5b50 0x7fb81c15d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.755+0000 7fb8242fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb81c15f380 0x7fb81c0ba730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.755+0000 7fb8242fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb81c15f760 0x7fb81c0bac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.755+0000 7fb8242fc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c0d1280 con 0x7fb81c15f380 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.755+0000 7fb8242fc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c0d1100 con 0x7fb81c0d5b50 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.755+0000 7fb8242fc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb81c0b9a90 con 0x7fb81c15f760 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.755+0000 7fb821897640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb81c15f380 0x7fb81c0ba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.755+0000 7fb821897640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb81c15f380 0x7fb81c0ba730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58020/0 (socket says 172.21.15.37:58020) 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.755+0000 7fb821897640 1 -- 172.21.15.37:0/126384967 learned_addr learned my addr 172.21.15.37:0/126384967 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb822098640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb81c0d5b50 0x7fb81c15d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb822899640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb81c15f760 0x7fb81c0bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb821897640 1 -- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb81c15f760 msgr2=0x7fb81c0bac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb821897640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb81c15f760 0x7fb81c0bac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb821897640 1 -- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb81c0d5b50 msgr2=0x7fb81c15d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb821897640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb81c0d5b50 0x7fb81c15d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb821897640 1 -- 172.21.15.37:0/126384967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c0b53c0 con 0x7fb81c15f380 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb821897640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb81c15f380 0x7fb81c0ba730 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb81000e960 tx=0x7fb81000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb81affd640 1 -- 172.21.15.37:0/126384967 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810017070 con 0x7fb81c15f380 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb81affd640 1 -- 172.21.15.37:0/126384967 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb81000ca00 con 0x7fb81c15f380 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb8242fc640 1 -- 172.21.15.37:0/126384967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81c0b4320 con 0x7fb81c15f380 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb81affd640 1 -- 172.21.15.37:0/126384967 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81000cdd0 con 0x7fb81c15f380 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.756+0000 7fb8242fc640 1 -- 172.21.15.37:0/126384967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81c0b48e0 con 0x7fb81c15f380 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.757+0000 7fb8242fc640 1 -- 172.21.15.37:0/126384967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb81c06a360 con 0x7fb81c15f380 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.758+0000 7fb81affd640 1 -- 172.21.15.37:0/126384967 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb81001a070 con 0x7fb81c15f380 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.758+0000 7fb81affd640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb80806c8f0 0x7fb80806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.758+0000 7fb81affd640 1 -- 172.21.15.37:0/126384967 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb81008bdc0 con 0x7fb81c15f380 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.758+0000 7fb822098640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb80806c8f0 0x7fb80806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.759+0000 7fb822098640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb80806c8f0 0x7fb80806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb814034000 tx=0x7fb81402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:29.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.762+0000 7fb81affd640 1 -- 172.21.15.37:0/126384967 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb81005a1f0 con 0x7fb81c15f380 2024-04-26T20:22:29.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.995+0000 7fb8242fc640 1 -- 172.21.15.37:0/126384967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb81c069560 con 0x7fb81c15f380 2024-04-26T20:22:29.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.995+0000 7fb81affd640 1 -- 172.21.15.37:0/126384967 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7fb810059af0 con 0x7fb81c15f380 2024-04-26T20:22:29.996 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:29.997 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2911741012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2911741012},{"type":"v1","addr":"172.21.15.134:6836","nonce":2911741012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1600483467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1600483467},{"type":"v1","addr":"172.21.15.37:6835","nonce":1600483467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2722614558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2722614558},{"type":"v1","addr":"172.21.15.134:6837","nonce":2722614558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:21:49.237855+0000","modified":"2024-04-26T20:21:52.255148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3920647125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3920647125},{"type":"v1","addr":"172.21.15.37:6834","nonce":3920647125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-04-26T20:22:29.997 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 -- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb80806c8f0 msgr2=0x7fb80806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb80806c8f0 0x7fb80806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb814034000 tx=0x7fb81402e000 comp rx=0 tx=0).stop 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 -- 172.21.15.37:0/126384967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb81c15f380 msgr2=0x7fb81c0ba730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb81c15f380 0x7fb81c0ba730 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb81000e960 tx=0x7fb81000ee20 comp rx=0 tx=0).stop 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 -- 172.21.15.37:0/126384967 shutdown_connections 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb80806c8f0 0x7fb80806edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb81c15f760 0x7fb81c0bac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb81c15f380 0x7fb81c0ba730 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 --2- 172.21.15.37:0/126384967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb81c0d5b50 0x7fb81c15d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 -- 172.21.15.37:0/126384967 >> 172.21.15.37:0/126384967 conn(0x7fb81c09e560 msgr2=0x7fb81c0b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 -- 172.21.15.37:0/126384967 shutdown_connections 2024-04-26T20:22:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:29.997+0000 7fb8242fc640 1 -- 172.21.15.37:0/126384967 wait complete. 2024-04-26T20:22:30.009 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.109+0000 7f08c6f19640 1 Processor -- start 2024-04-26T20:22:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.109+0000 7f08c6f19640 1 -- start start 2024-04-26T20:22:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08c6f19640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f08c00b3c90 0x7f08c0168150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08c6f19640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c00cf740 con 0x7f08c0160a00 2024-04-26T20:22:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08c6f19640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c00b2400 con 0x7f08c00b1f90 2024-04-26T20:22:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08c6f19640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f08c00d07e0 con 0x7f08c00b3c90 2024-04-26T20:22:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08c4cb5640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f08c0160a00 0x7f08c00cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57330/0 (socket says 172.21.15.37:57330) 2024-04-26T20:22:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08c4cb5640 1 -- 172.21.15.37:0/4262295345 learned_addr learned my addr 172.21.15.37:0/4262295345 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08c54b6640 1 --2- 172.21.15.37:0/4262295345 >> v2:172.21.15.134:3300/0 conn(0x7f08c00b3c90 0x7f08c0168150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607770936 0 0) 0x7f08c00cf740 con 0x7f08c0160a00 2024-04-26T20:22:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08b0003520 con 0x7f08c0160a00 2024-04-26T20:22:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2431393746 0 0) 0x7f08b0003520 con 0x7f08c0160a00 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 >> v2:172.21.15.134:3300/0 conn(0x7f08c00b3c90 msgr2=0x7f08c0168150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08bf7fe640 1 --2- 172.21.15.37:0/4262295345 >> v2:172.21.15.134:3300/0 conn(0x7f08c00b3c90 0x7f08c0168150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 >> v1:172.21.15.134:6789/0 conn(0x7f08c00b1f90 legacy=0x7f08c00d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c00cf080 con 0x7f08c0160a00 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f08b4001670 con 0x7f08c0160a00 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.110+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 >> v1:172.21.15.37:6789/0 conn(0x7f08c0160a00 legacy=0x7f08c00cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bf7fe640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b00039b0 0x7f08b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bf7fe640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b0004800 0x7f08b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bf7fe640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00051c0 0x7f08b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08c00d07e0 con 0x7f08b0004800 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08b000c070 con 0x7f08b00039b0 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f08b000c270 con 0x7f08b00051c0 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bffff640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b0004800 0x7f08b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08c4cb5640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b00039b0 0x7f08b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08c54b6640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00051c0 0x7f08b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bffff640 1 -- 172.21.15.37:0/4262295345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00051c0 msgr2=0x7f08b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bffff640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00051c0 0x7f08b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bffff640 1 -- 172.21.15.37:0/4262295345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b00039b0 msgr2=0x7f08b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bffff640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b00039b0 0x7f08b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08bffff640 1 -- 172.21.15.37:0/4262295345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b0003520 con 0x7f08b0004800 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08c54b6640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00051c0 0x7f08b000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.111+0000 7f08c54b6640 1 -- 172.21.15.37:0/4262295345 reap_dead start 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.112+0000 7f08bffff640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b0004800 0x7f08b0004c80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f08ac001750 tx=0x7f08b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77a3a007506c538 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.112+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac001370 con 0x7f08b0004800 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.112+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ac0052d0 con 0x7f08b0004800 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.112+0000 7f08bf7fe640 1 -- 172.21.15.37:0/4262295345 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac02abc0 con 0x7f08b0004800 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.112+0000 7f08c6f19640 1 -- 172.21.15.37:0/4262295345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b0004800 msgr2=0x7f08b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.112+0000 7f08c6f19640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b0004800 0x7f08b0004c80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f08ac001750 tx=0x7f08b0005830 comp rx=0 tx=0).stop 2024-04-26T20:22:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.112+0000 7f08c6f19640 1 -- 172.21.15.37:0/4262295345 shutdown_connections 2024-04-26T20:22:30.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.112+0000 7f08c6f19640 1 --2- 172.21.15.37:0/4262295345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b0004800 0x7f08b0004c80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.112+0000 7f08c6f19640 1 -- 172.21.15.37:0/4262295345 >> 172.21.15.37:0/4262295345 conn(0x7f08c009ba50 msgr2=0x7f08c009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:30.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.112+0000 7f08c6f19640 1 -- 172.21.15.37:0/4262295345 shutdown_connections 2024-04-26T20:22:30.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.112+0000 7f08c6f19640 1 -- 172.21.15.37:0/4262295345 wait complete. 2024-04-26T20:22:30.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.114+0000 7f08c6f19640 1 Processor -- start 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.114+0000 7f08c6f19640 1 -- start start 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.114+0000 7f08c6f19640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b0004800 0x7f08c01f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.114+0000 7f08c6f19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c01f8af0 0x7f08c00ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.114+0000 7f08c6f19640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c00cac80 0x7f08c01f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08c6f19640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08c00ceda0 con 0x7f08c01f8af0 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08c6f19640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08c00ceac0 con 0x7f08b0004800 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08c6f19640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f08c0160de0 con 0x7f08c00cac80 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c01f8af0 0x7f08c00ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c01f8af0 0x7f08c00ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58034/0 (socket says 172.21.15.37:58034) 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bffff640 1 -- 172.21.15.37:0/842021903 learned_addr learned my addr 172.21.15.37:0/842021903 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08c54b6640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c00cac80 0x7f08c01f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08c4cb5640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b0004800 0x7f08c01f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bffff640 1 -- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c00cac80 msgr2=0x7f08c01f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bffff640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c00cac80 0x7f08c01f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bffff640 1 -- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b0004800 msgr2=0x7f08c01f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bffff640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b0004800 0x7f08c01f85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bffff640 1 -- 172.21.15.37:0/842021903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c00cb470 con 0x7f08c01f8af0 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bffff640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c01f8af0 0x7f08c00ca740 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f08ac001870 tx=0x7f08ac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bd7fa640 1 -- 172.21.15.37:0/842021903 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac03f030 con 0x7f08c01f8af0 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bd7fa640 1 -- 172.21.15.37:0/842021903 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ac005480 con 0x7f08c01f8af0 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08c6f19640 1 -- 172.21.15.37:0/842021903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c00cb750 con 0x7f08c01f8af0 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08bd7fa640 1 -- 172.21.15.37:0/842021903 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac037040 con 0x7f08c01f8af0 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.115+0000 7f08c6f19640 1 -- 172.21.15.37:0/842021903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c01f63d0 con 0x7f08c01f8af0 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.116+0000 7f08c6f19640 1 -- 172.21.15.37:0/842021903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c00d23d0 con 0x7f08c01f8af0 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.117+0000 7f08bd7fa640 1 -- 172.21.15.37:0/842021903 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08ac042070 con 0x7f08c01f8af0 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.117+0000 7f08bd7fa640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f08a406c8f0 0x7f08a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.117+0000 7f08bd7fa640 1 -- 172.21.15.37:0/842021903 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f08ac0aaf10 con 0x7f08c01f8af0 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.117+0000 7f08c4cb5640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f08a406c8f0 0x7f08a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.118+0000 7f08c4cb5640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f08a406c8f0 0x7f08a406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cd2b0 tx=0x7f08b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:30.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.122+0000 7f08bd7fa640 1 -- 172.21.15.37:0/842021903 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08c00d23d0 con 0x7f08c01f8af0 2024-04-26T20:22:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.313+0000 7f08c6f19640 1 -- 172.21.15.37:0/842021903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08c006a360 con 0x7f08c01f8af0 2024-04-26T20:22:30.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.314+0000 7f08bd7fa640 1 -- 172.21.15.37:0/842021903 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16867 (secure 0 0 0) 0x7f08ac04a020 con 0x7f08c01f8af0 2024-04-26T20:22:30.315 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:30.315 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":21,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:22:30.240764+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 -- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f08a406c8f0 msgr2=0x7f08a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f08a406c8f0 0x7f08a406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cd2b0 tx=0x7f08b4009040 comp rx=0 tx=0).stop 2024-04-26T20:22:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 -- 172.21.15.37:0/842021903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c01f8af0 msgr2=0x7f08c00ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c01f8af0 0x7f08c00ca740 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f08ac001870 tx=0x7f08ac001120 comp rx=0 tx=0).stop 2024-04-26T20:22:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 -- 172.21.15.37:0/842021903 shutdown_connections 2024-04-26T20:22:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f08a406c8f0 0x7f08a406edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c00cac80 0x7f08c01f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c01f8af0 0x7f08c00ca740 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 --2- 172.21.15.37:0/842021903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b0004800 0x7f08c01f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 -- 172.21.15.37:0/842021903 >> 172.21.15.37:0/842021903 conn(0x7f08c009ba50 msgr2=0x7f08c015f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 -- 172.21.15.37:0/842021903 shutdown_connections 2024-04-26T20:22:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.316+0000 7f08c6f19640 1 -- 172.21.15.37:0/842021903 wait complete. 2024-04-26T20:22:30.328 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:22:30.329 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:22:30.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.428+0000 7fb6c7220640 1 Processor -- start 2024-04-26T20:22:30.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.428+0000 7fb6c7220640 1 -- start start 2024-04-26T20:22:30.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6c7220640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb6c0159a90 0x7fb6c006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6c7220640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb6c0070130 0x7fb6c015fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6c7220640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6c0072e30 con 0x7fb6c0160150 2024-04-26T20:22:30.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6c7220640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb6c01605c0 con 0x7fb6c0070130 2024-04-26T20:22:30.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6c7220640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb6c0073110 con 0x7fb6c0159a90 2024-04-26T20:22:30.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6b7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb6c0070130 0x7fb6c015fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6b7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb6c0070130 0x7fb6c015fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58040/0 (socket says 172.21.15.37:58040) 2024-04-26T20:22:30.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6b7fff640 1 -- 172.21.15.37:0/3588905986 learned_addr learned my addr 172.21.15.37:0/3588905986 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6c4fbc640 1 --2- 172.21.15.37:0/3588905986 >> v2:172.21.15.134:3300/0 conn(0x7fb6c0159a90 0x7fb6c006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6b7fff640 1 -- 172.21.15.37:0/3588905986 >> v2:172.21.15.134:3300/0 conn(0x7fb6c0159a90 msgr2=0x7fb6c006fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6b7fff640 1 --2- 172.21.15.37:0/3588905986 >> v2:172.21.15.134:3300/0 conn(0x7fb6c0159a90 0x7fb6c006fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6b7fff640 1 -- 172.21.15.37:0/3588905986 >> v1:172.21.15.134:6789/0 conn(0x7fb6c0160150 legacy=0x7fb6c015c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6b7fff640 1 -- 172.21.15.37:0/3588905986 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c00733f0 con 0x7fb6c0070130 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.429+0000 7fb6c4fbc640 1 --2- 172.21.15.37:0/3588905986 >> v2:172.21.15.134:3300/0 conn(0x7fb6c0159a90 0x7fb6c006fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.430+0000 7fb6b7fff640 1 --2- 172.21.15.37:0/3588905986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c0070130 0x7fb6c015fc10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0074a10 tx=0x7fb6b002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9025c84a006c22cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.430+0000 7fb6b77fe640 1 -- 172.21.15.37:0/3588905986 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0037070 con 0x7fb6c0070130 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.430+0000 7fb6b77fe640 1 -- 172.21.15.37:0/3588905986 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b002ae20 con 0x7fb6c0070130 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.430+0000 7fb6b77fe640 1 -- 172.21.15.37:0/3588905986 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0032660 con 0x7fb6c0070130 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.430+0000 7fb6c7220640 1 -- 172.21.15.37:0/3588905986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c0070130 msgr2=0x7fb6c015fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.430+0000 7fb6c7220640 1 --2- 172.21.15.37:0/3588905986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c0070130 0x7fb6c015fc10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0074a10 tx=0x7fb6b002a110 comp rx=0 tx=0).stop 2024-04-26T20:22:30.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.431+0000 7fb6c7220640 1 -- 172.21.15.37:0/3588905986 shutdown_connections 2024-04-26T20:22:30.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.431+0000 7fb6c7220640 1 --2- 172.21.15.37:0/3588905986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c0070130 0x7fb6c015fc10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.431+0000 7fb6c7220640 1 --2- 172.21.15.37:0/3588905986 >> v2:172.21.15.134:3300/0 conn(0x7fb6c0159a90 0x7fb6c006fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.431+0000 7fb6c7220640 1 -- 172.21.15.37:0/3588905986 >> 172.21.15.37:0/3588905986 conn(0x7fb6c00d0d50 msgr2=0x7fb6c00d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:30.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.431+0000 7fb6c7220640 1 -- 172.21.15.37:0/3588905986 shutdown_connections 2024-04-26T20:22:30.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.431+0000 7fb6c7220640 1 -- 172.21.15.37:0/3588905986 wait complete. 2024-04-26T20:22:30.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.432+0000 7fb6c7220640 1 Processor -- start 2024-04-26T20:22:30.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.432+0000 7fb6c7220640 1 -- start start 2024-04-26T20:22:30.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c7220640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6c0070130 0x7fb6c0148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c7220640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6c0159a90 0x7fb6c0154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c7220640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c0160150 0x7fb6c015e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c7220640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c0159e70 con 0x7fb6c0160150 2024-04-26T20:22:30.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c7220640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c01609d0 con 0x7fb6c0159a90 2024-04-26T20:22:30.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c7220640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c0159ff0 con 0x7fb6c0070130 2024-04-26T20:22:30.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c57bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c0160150 0x7fb6c015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c57bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c0160150 0x7fb6c015e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58052/0 (socket says 172.21.15.37:58052) 2024-04-26T20:22:30.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c57bd640 1 -- 172.21.15.37:0/3462597154 learned_addr learned my addr 172.21.15.37:0/3462597154 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:30.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6b7fff640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6c0159a90 0x7fb6c0154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c4fbc640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6c0070130 0x7fb6c0148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c57bd640 1 -- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6c0070130 msgr2=0x7fb6c0148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c57bd640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6c0070130 0x7fb6c0148850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c57bd640 1 -- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6c0159a90 msgr2=0x7fb6c0154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c57bd640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6c0159a90 0x7fb6c0154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c57bd640 1 -- 172.21.15.37:0/3462597154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c0155500 con 0x7fb6c0160150 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c57bd640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c0160150 0x7fb6c015e610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800b970 tx=0x7fb6b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6b57fa640 1 -- 172.21.15.37:0/3462597154 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8019070 con 0x7fb6c0160150 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6b57fa640 1 -- 172.21.15.37:0/3462597154 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b800cb40 con 0x7fb6c0160150 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6b57fa640 1 -- 172.21.15.37:0/3462597154 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8014440 con 0x7fb6c0160150 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c7220640 1 -- 172.21.15.37:0/3462597154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c01557e0 con 0x7fb6c0160150 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.433+0000 7fb6c7220640 1 -- 172.21.15.37:0/3462597154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c0155da0 con 0x7fb6c0160150 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.434+0000 7fb6c7220640 1 -- 172.21.15.37:0/3462597154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c006a360 con 0x7fb6c0160150 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.435+0000 7fb6b57fa640 1 -- 172.21.15.37:0/3462597154 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6b801c070 con 0x7fb6c0160150 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.435+0000 7fb6b57fa640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb6ac06c8f0 0x7fb6ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.435+0000 7fb6b57fa640 1 -- 172.21.15.37:0/3462597154 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7fb6b808b8a0 con 0x7fb6c0160150 2024-04-26T20:22:30.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.436+0000 7fb6c4fbc640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb6ac06c8f0 0x7fb6ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.436+0000 7fb6c4fbc640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb6ac06c8f0 0x7fb6ac06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc008850 tx=0x7fb6bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:30.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.439+0000 7fb6b57fa640 1 -- 172.21.15.37:0/3462597154 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb6b8059bc0 con 0x7fb6c0160150 2024-04-26T20:22:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.666+0000 7fb6c7220640 1 -- 172.21.15.37:0/3462597154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6c00d0730 con 0x7fb6c0160150 2024-04-26T20:22:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.667+0000 7fb6b57fa640 1 -- 172.21.15.37:0/3462597154 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7fb6b80594c0 con 0x7fb6c0160150 2024-04-26T20:22:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:30.668 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2911741012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2911741012},{"type":"v1","addr":"172.21.15.134:6836","nonce":2911741012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1600483467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1600483467},{"type":"v1","addr":"172.21.15.37:6835","nonce":1600483467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2722614558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2722614558},{"type":"v1","addr":"172.21.15.134:6837","nonce":2722614558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:21:49.237855+0000","modified":"2024-04-26T20:21:52.255148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3920647125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3920647125},{"type":"v1","addr":"172.21.15.37:6834","nonce":3920647125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-04-26T20:22:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-26T20:22:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 -- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb6ac06c8f0 msgr2=0x7fb6ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb6ac06c8f0 0x7fb6ac06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc008850 tx=0x7fb6bc008040 comp rx=0 tx=0).stop 2024-04-26T20:22:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 -- 172.21.15.37:0/3462597154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c0160150 msgr2=0x7fb6c015e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c0160150 0x7fb6c015e610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800b970 tx=0x7fb6b800be30 comp rx=0 tx=0).stop 2024-04-26T20:22:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 -- 172.21.15.37:0/3462597154 shutdown_connections 2024-04-26T20:22:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c0160150 0x7fb6c015e610 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6c0159a90 0x7fb6c0154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb6ac06c8f0 0x7fb6ac06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 --2- 172.21.15.37:0/3462597154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6c0070130 0x7fb6c0148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 -- 172.21.15.37:0/3462597154 >> 172.21.15.37:0/3462597154 conn(0x7fb6c00d0d50 msgr2=0x7fb6c00b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 -- 172.21.15.37:0/3462597154 shutdown_connections 2024-04-26T20:22:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.669+0000 7fb6c7220640 1 -- 172.21.15.37:0/3462597154 wait complete. 2024-04-26T20:22:30.681 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:22:30.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.780+0000 7f2bd81af640 1 Processor -- start 2024-04-26T20:22:30.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.780+0000 7f2bd81af640 1 -- start start 2024-04-26T20:22:30.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.781+0000 7f2bd81af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd00d1c50 0x7f2bd00c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.781+0000 7f2bd81af640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2bd00ce860 0x7f2bd00c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.781+0000 7f2bd81af640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bd00ced60 con 0x7f2bd015d310 2024-04-26T20:22:30.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.781+0000 7f2bd81af640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2bd00c6ab0 con 0x7f2bd00ce860 2024-04-26T20:22:30.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.781+0000 7f2bd81af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd00cf040 con 0x7f2bd00d1c50 2024-04-26T20:22:30.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.781+0000 7f2bd674c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2bd00ce860 0x7f2bd00c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.781+0000 7f2bd674c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2bd00ce860 0x7f2bd00c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58058/0 (socket says 172.21.15.37:58058) 2024-04-26T20:22:30.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.781+0000 7f2bd674c640 1 -- 172.21.15.37:0/2022915139 learned_addr learned my addr 172.21.15.37:0/2022915139 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:30.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.782+0000 7f2bd574a640 1 --2- 172.21.15.37:0/2022915139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd00d1c50 0x7f2bd00c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.782+0000 7f2bd674c640 1 -- 172.21.15.37:0/2022915139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd00d1c50 msgr2=0x7f2bd00c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.782+0000 7f2bd674c640 1 --2- 172.21.15.37:0/2022915139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd00d1c50 0x7f2bd00c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.782+0000 7f2bd674c640 1 -- 172.21.15.37:0/2022915139 >> v1:172.21.15.134:6789/0 conn(0x7f2bd015d310 legacy=0x7f2bd00d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.782+0000 7f2bd674c640 1 -- 172.21.15.37:0/2022915139 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd00cf320 con 0x7f2bd00ce860 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.782+0000 7f2bd674c640 1 --2- 172.21.15.37:0/2022915139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd00ce860 0x7f2bd00c7740 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009870 tx=0x7f2bc802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa6768b54cb1cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.783+0000 7f2bd4f49640 1 -- 172.21.15.37:0/2022915139 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8037070 con 0x7f2bd00ce860 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.783+0000 7f2bd4f49640 1 -- 172.21.15.37:0/2022915139 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc8032440 con 0x7f2bd00ce860 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.783+0000 7f2bd4f49640 1 -- 172.21.15.37:0/2022915139 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8032810 con 0x7f2bd00ce860 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.783+0000 7f2bd81af640 1 -- 172.21.15.37:0/2022915139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd00ce860 msgr2=0x7f2bd00c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.783+0000 7f2bd81af640 1 --2- 172.21.15.37:0/2022915139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd00ce860 0x7f2bd00c7740 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009870 tx=0x7f2bc802a210 comp rx=0 tx=0).stop 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.783+0000 7f2bd81af640 1 -- 172.21.15.37:0/2022915139 shutdown_connections 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.783+0000 7f2bd81af640 1 --2- 172.21.15.37:0/2022915139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd00ce860 0x7f2bd00c7740 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.783+0000 7f2bd81af640 1 --2- 172.21.15.37:0/2022915139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd00d1c50 0x7f2bd00c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.783+0000 7f2bd81af640 1 -- 172.21.15.37:0/2022915139 >> 172.21.15.37:0/2022915139 conn(0x7f2bd00b4a50 msgr2=0x7f2bd00b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:30.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.783+0000 7f2bd81af640 1 -- 172.21.15.37:0/2022915139 shutdown_connections 2024-04-26T20:22:30.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.783+0000 7f2bd81af640 1 -- 172.21.15.37:0/2022915139 wait complete. 2024-04-26T20:22:30.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.784+0000 7f2bd81af640 1 Processor -- start 2024-04-26T20:22:30.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.784+0000 7f2bd81af640 1 -- start start 2024-04-26T20:22:30.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd81af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd00ce860 0x7f2bd014e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd81af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd00d1c50 0x7f2bd00cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd81af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd015d310 0x7f2bd00c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd81af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd00b5970 con 0x7f2bd015d310 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd81af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd00b57f0 con 0x7f2bd00d1c50 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd81af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd015d6f0 con 0x7f2bd00ce860 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd674c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd015d310 0x7f2bd00c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd674c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd015d310 0x7f2bd00c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58074/0 (socket says 172.21.15.37:58074) 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd674c640 1 -- 172.21.15.37:0/126164293 learned_addr learned my addr 172.21.15.37:0/126164293 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd5f4b640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd00ce860 0x7f2bd014e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd574a640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd00d1c50 0x7f2bd00cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd674c640 1 -- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd00ce860 msgr2=0x7f2bd014e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd674c640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd00ce860 0x7f2bd014e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd674c640 1 -- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd00d1c50 msgr2=0x7f2bd00cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd674c640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd00d1c50 0x7f2bd00cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.785+0000 7f2bd674c640 1 -- 172.21.15.37:0/126164293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd0157f20 con 0x7f2bd015d310 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.786+0000 7f2bd674c640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd015d310 0x7f2bd00c8c70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2bc802a250 tx=0x7f2bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.786+0000 7f2bce7fc640 1 -- 172.21.15.37:0/126164293 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8041020 con 0x7f2bd015d310 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.786+0000 7f2bce7fc640 1 -- 172.21.15.37:0/126164293 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc80306c0 con 0x7f2bd015d310 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.786+0000 7f2bd81af640 1 -- 172.21.15.37:0/126164293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd0158200 con 0x7f2bd015d310 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.786+0000 7f2bce7fc640 1 -- 172.21.15.37:0/126164293 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8037040 con 0x7f2bd015d310 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.786+0000 7f2bd81af640 1 -- 172.21.15.37:0/126164293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd01587c0 con 0x7f2bd015d310 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.787+0000 7f2bd81af640 1 -- 172.21.15.37:0/126164293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd00693e0 con 0x7f2bd015d310 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.787+0000 7f2bce7fc640 1 -- 172.21.15.37:0/126164293 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bc8044020 con 0x7f2bd015d310 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.788+0000 7f2bce7fc640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.788+0000 7f2bce7fc640 1 -- 172.21.15.37:0/126164293 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f2bc80ac350 con 0x7f2bd015d310 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.788+0000 7f2bd5f4b640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.788+0000 7f2bd5f4b640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009f20 tx=0x7f2bc00094d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:30.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:30.792+0000 7f2bce7fc640 1 -- 172.21.15.37:0/126164293 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2bc804b030 con 0x7f2bd015d310 2024-04-26T20:22:31.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.014+0000 7f2bd81af640 1 -- 172.21.15.37:0/126164293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bd00c71c0 con 0x7f2bd015d310 2024-04-26T20:22:31.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.015+0000 7f2bce7fc640 1 -- 172.21.15.37:0/126164293 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7f2bc807a790 con 0x7f2bd015d310 2024-04-26T20:22:31.016 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:31.016 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2911741012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2911741012},{"type":"v1","addr":"172.21.15.134:6836","nonce":2911741012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1600483467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1600483467},{"type":"v1","addr":"172.21.15.37:6835","nonce":1600483467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2722614558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2722614558},{"type":"v1","addr":"172.21.15.134:6837","nonce":2722614558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:21:49.237855+0000","modified":"2024-04-26T20:21:52.255148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3920647125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3920647125},{"type":"v1","addr":"172.21.15.37:6834","nonce":3920647125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-04-26T20:22:31.016 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-26T20:22:31.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.016+0000 7f2bd81af640 1 -- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bbc06c8f0 msgr2=0x7f2bbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.016+0000 7f2bd81af640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009f20 tx=0x7f2bc00094d0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.016+0000 7f2bd81af640 1 -- 172.21.15.37:0/126164293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd015d310 msgr2=0x7f2bd00c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.016+0000 7f2bd81af640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd015d310 0x7f2bd00c8c70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2bc802a250 tx=0x7f2bc80046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.017+0000 7f2bd81af640 1 -- 172.21.15.37:0/126164293 shutdown_connections 2024-04-26T20:22:31.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.017+0000 7f2bd81af640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd015d310 0x7f2bd00c8c70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.017+0000 7f2bd81af640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd00d1c50 0x7f2bd00cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.017+0000 7f2bd81af640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.017+0000 7f2bd81af640 1 --2- 172.21.15.37:0/126164293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd00ce860 0x7f2bd014e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.017+0000 7f2bd81af640 1 -- 172.21.15.37:0/126164293 >> 172.21.15.37:0/126164293 conn(0x7f2bd00b4a50 msgr2=0x7f2bd00cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:31.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.017+0000 7f2bd81af640 1 -- 172.21.15.37:0/126164293 shutdown_connections 2024-04-26T20:22:31.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.017+0000 7f2bd81af640 1 -- 172.21.15.37:0/126164293 wait complete. 2024-04-26T20:22:31.029 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:31.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.127+0000 7f5abbeea640 1 Processor -- start 2024-04-26T20:22:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.128+0000 7f5abbeea640 1 -- start start 2024-04-26T20:22:31.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.128+0000 7f5abbeea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab40d0540 0x7f5ab4072a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.128+0000 7f5abbeea640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab415be00 con 0x7f5ab406f490 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.128+0000 7f5abbeea640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab415c0a0 con 0x7f5ab4072f50 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.128+0000 7f5abbeea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab415c380 con 0x7f5ab40d0540 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.128+0000 7f5aba487640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5ab406f490 0x7f5ab40cead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57342/0 (socket says 172.21.15.37:57342) 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.128+0000 7f5aba487640 1 -- 172.21.15.37:0/3241743675 learned_addr learned my addr 172.21.15.37:0/3241743675 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab9c86640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab40d0540 0x7f5ab4072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416100272 0 0) 0x7f5ab415be00 con 0x7f5ab406f490 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a9c005730 con 0x7f5ab406f490 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186664412 0 0) 0x7f5ab415c0a0 con 0x7f5ab4072f50 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ab415be00 con 0x7f5ab4072f50 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3928481694 0 0) 0x7f5a9c005730 con 0x7f5ab406f490 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab40d0540 msgr2=0x7f5ab4072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab40d0540 0x7f5ab4072a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 >> v1:172.21.15.134:6789/0 conn(0x7f5ab4072f50 legacy=0x7f5ab406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab415c660 con 0x7f5ab406f490 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f5aa40019f0 con 0x7f5ab406f490 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 >> v1:172.21.15.37:6789/0 conn(0x7f5ab406f490 legacy=0x7f5ab40cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab40d0540 0x7f5a9c005d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c0062e0 0x7f5a9c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 0x7f5a9c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab415c380 con 0x7f5a9c006f50 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c00dcb0 con 0x7f5a9c0062e0 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9c00deb0 con 0x7f5ab40d0540 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5aba487640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 0x7f5a9c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab9485640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c0062e0 0x7f5a9c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.129+0000 7f5ab9c86640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab40d0540 0x7f5a9c005d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.130+0000 7f5aba487640 1 -- 172.21.15.37:0/3241743675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab40d0540 msgr2=0x7f5a9c005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.130+0000 7f5aba487640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab40d0540 0x7f5a9c005d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.130+0000 7f5aba487640 1 -- 172.21.15.37:0/3241743675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c0062e0 msgr2=0x7f5a9c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:31.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.130+0000 7f5aba487640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c0062e0 0x7f5a9c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.130+0000 7f5aba487640 1 -- 172.21.15.37:0/3241743675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c005730 con 0x7f5a9c006f50 2024-04-26T20:22:31.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.130+0000 7f5aba487640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 0x7f5a9c0073d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400a7f0 tx=0x7f5a9c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e059d960d44f80c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:31.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.130+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400a410 con 0x7f5a9c006f50 2024-04-26T20:22:31.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.130+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa402f070 con 0x7f5a9c006f50 2024-04-26T20:22:31.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.130+0000 7f5ab8c84640 1 -- 172.21.15.37:0/3241743675 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa402bc70 con 0x7f5a9c006f50 2024-04-26T20:22:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.131+0000 7f5abbeea640 1 -- 172.21.15.37:0/3241743675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 msgr2=0x7f5a9c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.131+0000 7f5abbeea640 1 --2- 172.21.15.37:0/3241743675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 0x7f5a9c0073d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400a7f0 tx=0x7f5a9c007a40 comp rx=0 tx=0).stop 2024-04-26T20:22:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.131+0000 7f5aba487640 1 -- 172.21.15.37:0/3241743675 reap_dead start 2024-04-26T20:22:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.131+0000 7f5abbeea640 1 -- 172.21.15.37:0/3241743675 shutdown_connections 2024-04-26T20:22:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.131+0000 7f5abbeea640 1 -- 172.21.15.37:0/3241743675 >> 172.21.15.37:0/3241743675 conn(0x7f5ab40d0d50 msgr2=0x7f5ab40d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.131+0000 7f5abbeea640 1 -- 172.21.15.37:0/3241743675 shutdown_connections 2024-04-26T20:22:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.131+0000 7f5abbeea640 1 -- 172.21.15.37:0/3241743675 wait complete. 2024-04-26T20:22:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.132+0000 7f5abbeea640 1 Processor -- start 2024-04-26T20:22:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.132+0000 7f5abbeea640 1 -- start start 2024-04-26T20:22:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5abbeea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 0x7f5ab40aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5abbeea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab415a5a0 0x7f5ab415aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5abbeea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab4149520 0x7f5ab41499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5abbeea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40d0aa0 con 0x7f5a9c006f50 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5abbeea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40d0920 con 0x7f5ab415a5a0 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5abbeea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab4072a10 con 0x7f5ab4149520 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5ab9c86640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 0x7f5ab40aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5ab9c86640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 0x7f5ab40aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58094/0 (socket says 172.21.15.37:58094) 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5ab9c86640 1 -- 172.21.15.37:0/2146763800 learned_addr learned my addr 172.21.15.37:0/2146763800 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5ab9485640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab415a5a0 0x7f5ab415aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5aba487640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab4149520 0x7f5ab41499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5ab9c86640 1 -- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab4149520 msgr2=0x7f5ab41499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5ab9c86640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab4149520 0x7f5ab41499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5ab9c86640 1 -- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab415a5a0 msgr2=0x7f5ab415aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5ab9c86640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab415a5a0 0x7f5ab415aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.133+0000 7f5ab9c86640 1 -- 172.21.15.37:0/2146763800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab40aa650 con 0x7f5a9c006f50 2024-04-26T20:22:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.134+0000 7f5ab9c86640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 0x7f5ab40aec00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0119d0 tx=0x7f5aac011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.134+0000 7f5ab27fc640 1 -- 172.21.15.37:0/2146763800 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00faf0 con 0x7f5a9c006f50 2024-04-26T20:22:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.134+0000 7f5ab27fc640 1 -- 172.21.15.37:0/2146763800 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aac00fc90 con 0x7f5a9c006f50 2024-04-26T20:22:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.134+0000 7f5abbeea640 1 -- 172.21.15.37:0/2146763800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab40aa930 con 0x7f5a9c006f50 2024-04-26T20:22:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.134+0000 7f5ab27fc640 1 -- 172.21.15.37:0/2146763800 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac014440 con 0x7f5a9c006f50 2024-04-26T20:22:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.134+0000 7f5abbeea640 1 -- 172.21.15.37:0/2146763800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab40aaef0 con 0x7f5a9c006f50 2024-04-26T20:22:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.135+0000 7f5abbeea640 1 -- 172.21.15.37:0/2146763800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab406a360 con 0x7f5a9c006f50 2024-04-26T20:22:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.135+0000 7f5ab27fc640 1 -- 172.21.15.37:0/2146763800 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5aac012040 con 0x7f5a9c006f50 2024-04-26T20:22:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.136+0000 7f5ab27fc640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5aa006c8f0 0x7f5aa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.136+0000 7f5ab27fc640 1 -- 172.21.15.37:0/2146763800 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f5aac08c0e0 con 0x7f5a9c006f50 2024-04-26T20:22:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.136+0000 7f5ab9485640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5aa006c8f0 0x7f5aa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.136+0000 7f5ab9485640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5aa006c8f0 0x7f5aa006edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40b6e00 tx=0x7f5aa8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.140+0000 7f5ab27fc640 1 -- 172.21.15.37:0/2146763800 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5aac019030 con 0x7f5a9c006f50 2024-04-26T20:22:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.328+0000 7f5abbeea640 1 -- 172.21.15.37:0/2146763800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ab406f8e0 con 0x7f5a9c006f50 2024-04-26T20:22:31.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.330+0000 7f5ab27fc640 1 -- 172.21.15.37:0/2146763800 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16867 (secure 0 0 0) 0x7f5aac05a520 con 0x7f5a9c006f50 2024-04-26T20:22:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:31.331 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:22:31.245137+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:21:47.476747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:21:48.186397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:31.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.331+0000 7f5abbeea640 1 -- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5aa006c8f0 msgr2=0x7f5aa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.331+0000 7f5abbeea640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5aa006c8f0 0x7f5aa006edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40b6e00 tx=0x7f5aa8009040 comp rx=0 tx=0).stop 2024-04-26T20:22:31.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.332+0000 7f5abbeea640 1 -- 172.21.15.37:0/2146763800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 msgr2=0x7f5ab40aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.332+0000 7f5abbeea640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 0x7f5ab40aec00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0119d0 tx=0x7f5aac011dd0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.332+0000 7f5abbeea640 1 -- 172.21.15.37:0/2146763800 shutdown_connections 2024-04-26T20:22:31.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.332+0000 7f5abbeea640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5aa006c8f0 0x7f5aa006edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.332+0000 7f5abbeea640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ab4149520 0x7f5ab41499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.332+0000 7f5abbeea640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab415a5a0 0x7f5ab415aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.332+0000 7f5abbeea640 1 --2- 172.21.15.37:0/2146763800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a9c006f50 0x7f5ab40aec00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.332+0000 7f5abbeea640 1 -- 172.21.15.37:0/2146763800 >> 172.21.15.37:0/2146763800 conn(0x7f5ab40d0d50 msgr2=0x7f5ab40b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:31.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.332+0000 7f5abbeea640 1 -- 172.21.15.37:0/2146763800 shutdown_connections 2024-04-26T20:22:31.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.332+0000 7f5abbeea640 1 -- 172.21.15.37:0/2146763800 wait complete. 2024-04-26T20:22:31.345 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:22:31.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.447+0000 7f504c5e4640 1 Processor -- start 2024-04-26T20:22:31.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.447+0000 7f504c5e4640 1 -- start start 2024-04-26T20:22:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504c5e4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f504415f760 0x7f50440b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504c5e4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50440d3de0 con 0x7f50440d22e0 2024-04-26T20:22:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504c5e4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50440d40c0 con 0x7f504415f380 2024-04-26T20:22:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504c5e4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f50440d43a0 con 0x7f504415f760 2024-04-26T20:22:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f5049b7f640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f50440d22e0 0x7f50440b8a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57348/0 (socket says 172.21.15.37:57348) 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f5049b7f640 1 -- 172.21.15.37:0/3050316168 learned_addr learned my addr 172.21.15.37:0/3050316168 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 675022811 0 0) 0x7f50440d3de0 con 0x7f50440d22e0 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5030003520 con 0x7f50440d22e0 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504ab81640 1 --2- 172.21.15.37:0/3050316168 >> v2:172.21.15.134:3300/0 conn(0x7f504415f760 0x7f50440b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1823053860 0 0) 0x7f5030003520 con 0x7f50440d22e0 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 >> v2:172.21.15.134:3300/0 conn(0x7f504415f760 msgr2=0x7f50440b9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504937e640 1 --2- 172.21.15.37:0/3050316168 >> v2:172.21.15.134:3300/0 conn(0x7f504415f760 0x7f50440b9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 >> v1:172.21.15.134:6789/0 conn(0x7f504415f380 legacy=0x7f50440d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50440b8280 con 0x7f50440d22e0 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.448+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f5034001670 con 0x7f50440d22e0 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 >> v1:172.21.15.37:6789/0 conn(0x7f50440d22e0 legacy=0x7f50440b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f504937e640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50300039e0 0x7f5030003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f504937e640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5030004800 0x7f5030004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f504937e640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50300051c0 0x7f503000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50440d43a0 con 0x7f5030004800 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f503000c0a0 con 0x7f50300039e0 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f503000c2a0 con 0x7f50300051c0 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f5049b7f640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5030004800 0x7f5030004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f504a380640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50300039e0 0x7f5030003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f504ab81640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50300051c0 0x7f503000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f5049b7f640 1 -- 172.21.15.37:0/3050316168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50300051c0 msgr2=0x7f503000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f5049b7f640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50300051c0 0x7f503000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f5049b7f640 1 -- 172.21.15.37:0/3050316168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50300039e0 msgr2=0x7f5030003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f5049b7f640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50300039e0 0x7f5030003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f5049b7f640 1 -- 172.21.15.37:0/3050316168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5030003520 con 0x7f5030004800 2024-04-26T20:22:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f504ab81640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50300051c0 0x7f503000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.449+0000 7f504ab81640 1 -- 172.21.15.37:0/3050316168 reap_dead start 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.450+0000 7f5049b7f640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5030004800 0x7f5030004c80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f503400c210 tx=0x7f5030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cb074a856e04c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.450+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034007ce0 con 0x7f5030004800 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.450+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5034005ad0 con 0x7f5030004800 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.450+0000 7f504937e640 1 -- 172.21.15.37:0/3050316168 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034007070 con 0x7f5030004800 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.450+0000 7f504c5e4640 1 -- 172.21.15.37:0/3050316168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5030004800 msgr2=0x7f5030004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.450+0000 7f504c5e4640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5030004800 0x7f5030004c80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f503400c210 tx=0x7f5030005830 comp rx=0 tx=0).stop 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.451+0000 7f504c5e4640 1 -- 172.21.15.37:0/3050316168 shutdown_connections 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.451+0000 7f504c5e4640 1 --2- 172.21.15.37:0/3050316168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5030004800 0x7f5030004c80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.451+0000 7f504c5e4640 1 -- 172.21.15.37:0/3050316168 >> 172.21.15.37:0/3050316168 conn(0x7f504409e560 msgr2=0x7f50440d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.451+0000 7f504c5e4640 1 -- 172.21.15.37:0/3050316168 shutdown_connections 2024-04-26T20:22:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.451+0000 7f504c5e4640 1 -- 172.21.15.37:0/3050316168 wait complete. 2024-04-26T20:22:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.452+0000 7f504c5e4640 1 Processor -- start 2024-04-26T20:22:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.452+0000 7f504c5e4640 1 -- start start 2024-04-26T20:22:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504c5e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5030004800 0x7f50441f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504c5e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5044158af0 0x7f5044161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504c5e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5044162150 0x7f50440cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504c5e4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50440d1280 con 0x7f5044158af0 2024-04-26T20:22:31.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504c5e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f50440d1100 con 0x7f5044162150 2024-04-26T20:22:31.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504c5e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f50440d5b50 con 0x7f5030004800 2024-04-26T20:22:31.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f5049b7f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5044158af0 0x7f5044161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f5049b7f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5044158af0 0x7f5044161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58116/0 (socket says 172.21.15.37:58116) 2024-04-26T20:22:31.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f5049b7f640 1 -- 172.21.15.37:0/4071495466 learned_addr learned my addr 172.21.15.37:0/4071495466 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:31.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504ab81640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5044162150 0x7f50440cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504a380640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5030004800 0x7f50441f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f5049b7f640 1 -- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5030004800 msgr2=0x7f50441f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f5049b7f640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5030004800 0x7f50441f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f5049b7f640 1 -- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5044162150 msgr2=0x7f50440cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f5049b7f640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5044162150 0x7f50440cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f5049b7f640 1 -- 172.21.15.37:0/4071495466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50440c9a00 con 0x7f5044158af0 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504ab81640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5044162150 0x7f50440cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504a380640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5030004800 0x7f50441f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f5049b7f640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5044158af0 0x7f5044161c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f503400c330 tx=0x7f50440b8450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f503affd640 1 -- 172.21.15.37:0/4071495466 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034042070 con 0x7f5044158af0 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f503affd640 1 -- 172.21.15.37:0/4071495466 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5034001630 con 0x7f5044158af0 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f503affd640 1 -- 172.21.15.37:0/4071495466 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034037040 con 0x7f5044158af0 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504c5e4640 1 -- 172.21.15.37:0/4071495466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50440c9f20 con 0x7f5044158af0 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.453+0000 7f504c5e4640 1 -- 172.21.15.37:0/4071495466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50440ca4e0 con 0x7f5044158af0 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.454+0000 7f504c5e4640 1 -- 172.21.15.37:0/4071495466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504406a360 con 0x7f5044158af0 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.455+0000 7f503affd640 1 -- 172.21.15.37:0/4071495466 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50340049e0 con 0x7f5044158af0 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.455+0000 7f503affd640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f502806c920 0x7f502806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.455+0000 7f503affd640 1 -- 172.21.15.37:0/4071495466 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f50340ab160 con 0x7f5044158af0 2024-04-26T20:22:31.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.455+0000 7f504a380640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f502806c920 0x7f502806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.456+0000 7f504a380640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f502806c920 0x7f502806ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5040000c00 tx=0x7f5040009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:31.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.460+0000 7f503affd640 1 -- 172.21.15.37:0/4071495466 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5034078ec0 con 0x7f5044158af0 2024-04-26T20:22:31.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.686+0000 7f504c5e4640 1 -- 172.21.15.37:0/4071495466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f50440d2fa0 con 0x7f5044158af0 2024-04-26T20:22:31.687 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-26T20:22:31.687+0000 7f284b532640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:22:31.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.688+0000 7f503affd640 1 -- 172.21.15.37:0/4071495466 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f503407d1c0 con 0x7f5044158af0 2024-04-26T20:22:31.689 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:22:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.691+0000 7f504c5e4640 1 -- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f502806c920 msgr2=0x7f502806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.691+0000 7f504c5e4640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f502806c920 0x7f502806ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5040000c00 tx=0x7f5040009040 comp rx=0 tx=0).stop 2024-04-26T20:22:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.692+0000 7f504c5e4640 1 -- 172.21.15.37:0/4071495466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5044158af0 msgr2=0x7f5044161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.692+0000 7f504c5e4640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5044158af0 0x7f5044161c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f503400c330 tx=0x7f50440b8450 comp rx=0 tx=0).stop 2024-04-26T20:22:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.692+0000 7f504c5e4640 1 -- 172.21.15.37:0/4071495466 shutdown_connections 2024-04-26T20:22:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.692+0000 7f504c5e4640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5044162150 0x7f50440cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.692+0000 7f504c5e4640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5044158af0 0x7f5044161c10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.692+0000 7f504c5e4640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f502806c920 0x7f502806ede0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.692+0000 7f504c5e4640 1 --2- 172.21.15.37:0/4071495466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5030004800 0x7f50441f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.692+0000 7f504c5e4640 1 -- 172.21.15.37:0/4071495466 >> 172.21.15.37:0/4071495466 conn(0x7f504409e560 msgr2=0x7f50440cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.692+0000 7f504c5e4640 1 -- 172.21.15.37:0/4071495466 shutdown_connections 2024-04-26T20:22:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.692+0000 7f504c5e4640 1 -- 172.21.15.37:0/4071495466 wait complete. 2024-04-26T20:22:31.695 INFO:tasks.ceph.mds.a.smithi037.stderr: -7523> 2024-04-26T20:21:52.961+0000 7f8519c9a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:22:31.705 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:22:31.728 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-26T20:22:31.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.807+0000 7ff1412a9640 1 Processor -- start 2024-04-26T20:22:31.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.807+0000 7ff1412a9640 1 -- start start 2024-04-26T20:22:31.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.808+0000 7ff1412a9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff13c0b5510 0x7ff13c0d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.808+0000 7ff1412a9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff13c0d3d60 con 0x7ff13c0b58f0 2024-04-26T20:22:31.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.808+0000 7ff1412a9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff13c0c9cc0 con 0x7ff13c0b6820 2024-04-26T20:22:31.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.808+0000 7ff1412a9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff13c0b5d80 con 0x7ff13c0b5510 2024-04-26T20:22:31.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.808+0000 7ff13b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff13c0b58f0 0x7ff13c0d3650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57376/0 (socket says 172.21.15.37:57376) 2024-04-26T20:22:31.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.808+0000 7ff13b7fe640 1 -- 172.21.15.37:0/1124580638 learned_addr learned my addr 172.21.15.37:0/1124580638 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:31.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 204678023 0 0) 0x7ff13c0d3d60 con 0x7ff13c0b58f0 2024-04-26T20:22:31.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff13affd640 1 --2- 172.21.15.37:0/1124580638 >> v2:172.21.15.134:3300/0 conn(0x7ff13c0b5510 0x7ff13c0d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff124003520 con 0x7ff13c0b58f0 2024-04-26T20:22:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 381110355 0 0) 0x7ff124003520 con 0x7ff13c0b58f0 2024-04-26T20:22:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 >> v2:172.21.15.134:3300/0 conn(0x7ff13c0b5510 msgr2=0x7ff13c0d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 --2- 172.21.15.37:0/1124580638 >> v2:172.21.15.134:3300/0 conn(0x7ff13c0b5510 0x7ff13c0d1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 >> v1:172.21.15.134:6789/0 conn(0x7ff13c0b6820 legacy=0x7ff13c0d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c0b5fd0 con 0x7ff13c0b58f0 2024-04-26T20:22:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7ff12c0019f0 con 0x7ff13c0b58f0 2024-04-26T20:22:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 >> v1:172.21.15.37:6789/0 conn(0x7ff13c0b58f0 legacy=0x7ff13c0d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 --2- 172.21.15.37:0/1124580638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1240039b0 0x7ff124003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 --2- 172.21.15.37:0/1124580638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff124004800 0x7ff124004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 --2- 172.21.15.37:0/1124580638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff124005210 0x7ff12400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c0b5d80 con 0x7ff124004800 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff12400c140 con 0x7ff124005210 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff12400c340 con 0x7ff1240039b0 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff13a7fc640 1 --2- 172.21.15.37:0/1124580638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff124004800 0x7ff124004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff13affd640 1 --2- 172.21.15.37:0/1124580638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1240039b0 0x7ff124003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff13b7fe640 1 --2- 172.21.15.37:0/1124580638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff124005210 0x7ff12400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff13a7fc640 1 -- 172.21.15.37:0/1124580638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1240039b0 msgr2=0x7ff124003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff13a7fc640 1 --2- 172.21.15.37:0/1124580638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1240039b0 0x7ff124003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff13a7fc640 1 -- 172.21.15.37:0/1124580638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff124005210 msgr2=0x7ff12400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.809+0000 7ff13a7fc640 1 --2- 172.21.15.37:0/1124580638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff124005210 0x7ff12400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.810+0000 7ff13b7fe640 1 -- 172.21.15.37:0/1124580638 reap_dead start 2024-04-26T20:22:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.810+0000 7ff13a7fc640 1 -- 172.21.15.37:0/1124580638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff124003520 con 0x7ff124004800 2024-04-26T20:22:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.810+0000 7ff13a7fc640 1 --2- 172.21.15.37:0/1124580638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff124004800 0x7ff124004c80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff130001750 tx=0x7ff124005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727ef3278f213d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.810+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130001370 con 0x7ff124004800 2024-04-26T20:22:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.810+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1300052d0 con 0x7ff124004800 2024-04-26T20:22:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.810+0000 7ff139ffb640 1 -- 172.21.15.37:0/1124580638 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13002abc0 con 0x7ff124004800 2024-04-26T20:22:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.811+0000 7ff1412a9640 1 -- 172.21.15.37:0/1124580638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff124004800 msgr2=0x7ff124004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.811+0000 7ff1412a9640 1 --2- 172.21.15.37:0/1124580638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff124004800 0x7ff124004c80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff130001750 tx=0x7ff124005830 comp rx=0 tx=0).stop 2024-04-26T20:22:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.811+0000 7ff1412a9640 1 -- 172.21.15.37:0/1124580638 shutdown_connections 2024-04-26T20:22:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.811+0000 7ff1412a9640 1 --2- 172.21.15.37:0/1124580638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff124004800 0x7ff124004c80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.811+0000 7ff1412a9640 1 -- 172.21.15.37:0/1124580638 >> 172.21.15.37:0/1124580638 conn(0x7ff13c0d5160 msgr2=0x7ff13c0d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.811+0000 7ff1412a9640 1 -- 172.21.15.37:0/1124580638 shutdown_connections 2024-04-26T20:22:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.811+0000 7ff1412a9640 1 -- 172.21.15.37:0/1124580638 wait complete. 2024-04-26T20:22:31.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.812+0000 7ff1412a9640 1 Processor -- start 2024-04-26T20:22:31.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.812+0000 7ff1412a9640 1 -- start start 2024-04-26T20:22:31.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff1412a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff124004800 0x7ff13c0cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff1412a9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c1f9010 0x7ff13c1ea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff1412a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c15bd80 0x7ff13c15c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff1412a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c0d0880 con 0x7ff13c15bd80 2024-04-26T20:22:31.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff1412a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c0d5f60 con 0x7ff124004800 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff1412a9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff13c0d0b60 con 0x7ff13c1f9010 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c15bd80 0x7ff13c15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c15bd80 0x7ff13c15c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58148/0 (socket says 172.21.15.37:58148) 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff13b7fe640 1 -- 172.21.15.37:0/2623596870 learned_addr learned my addr 172.21.15.37:0/2623596870 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff13affd640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff124004800 0x7ff13c0cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff13a7fc640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c1f9010 0x7ff13c1ea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff13b7fe640 1 -- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c1f9010 msgr2=0x7ff13c1ea3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff13b7fe640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c1f9010 0x7ff13c1ea3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff13b7fe640 1 -- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff124004800 msgr2=0x7ff13c0cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff13b7fe640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff124004800 0x7ff13c0cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.813+0000 7ff13b7fe640 1 -- 172.21.15.37:0/2623596870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c1f9490 con 0x7ff13c15bd80 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.814+0000 7ff13b7fe640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c15bd80 0x7ff13c15c200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff12c001f00 tx=0x7ff12c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.814+0000 7ff1237fe640 1 -- 172.21.15.37:0/2623596870 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c017020 con 0x7ff13c15bd80 2024-04-26T20:22:31.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.814+0000 7ff1237fe640 1 -- 172.21.15.37:0/2623596870 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12c00abc0 con 0x7ff13c15bd80 2024-04-26T20:22:31.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.814+0000 7ff1412a9640 1 -- 172.21.15.37:0/2623596870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c0b3ab0 con 0x7ff13c15bd80 2024-04-26T20:22:31.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.814+0000 7ff1237fe640 1 -- 172.21.15.37:0/2623596870 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0133f0 con 0x7ff13c15bd80 2024-04-26T20:22:31.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.814+0000 7ff1412a9640 1 -- 172.21.15.37:0/2623596870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c0b4070 con 0x7ff13c15bd80 2024-04-26T20:22:31.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.815+0000 7ff1412a9640 1 -- 172.21.15.37:0/2623596870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff13c0693e0 con 0x7ff13c15bd80 2024-04-26T20:22:31.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.815+0000 7ff1237fe640 1 -- 172.21.15.37:0/2623596870 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff12c01a020 con 0x7ff13c15bd80 2024-04-26T20:22:31.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.816+0000 7ff1237fe640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff12806c920 0x7ff12806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.816+0000 7ff1237fe640 1 -- 172.21.15.37:0/2623596870 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7ff12c08c2c0 con 0x7ff13c15bd80 2024-04-26T20:22:31.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.816+0000 7ff13affd640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff12806c920 0x7ff12806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.816+0000 7ff13affd640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff12806c920 0x7ff12806ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff134009b00 tx=0x7ff134008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:31.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:31.820+0000 7ff1237fe640 1 -- 172.21.15.37:0/2623596870 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff12c021020 con 0x7ff13c15bd80 2024-04-26T20:22:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.044+0000 7ff1412a9640 1 -- 172.21.15.37:0/2623596870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff13c0cf690 con 0x7ff13c15bd80 2024-04-26T20:22:32.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.690+0000 7ff1237fe640 1 -- 172.21.15.37:0/2623596870 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7ff12c05a6b0 con 0x7ff13c15bd80 2024-04-26T20:22:32.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.692+0000 7ff1412a9640 1 -- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff12806c920 msgr2=0x7ff12806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:32.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.692+0000 7ff1412a9640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff12806c920 0x7ff12806ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff134009b00 tx=0x7ff134008040 comp rx=0 tx=0).stop 2024-04-26T20:22:32.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.692+0000 7ff1412a9640 1 -- 172.21.15.37:0/2623596870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c15bd80 msgr2=0x7ff13c15c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:32.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.692+0000 7ff1412a9640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c15bd80 0x7ff13c15c200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff12c001f00 tx=0x7ff12c007bd0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.692+0000 7ff1412a9640 1 -- 172.21.15.37:0/2623596870 shutdown_connections 2024-04-26T20:22:32.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.692+0000 7ff1412a9640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c15bd80 0x7ff13c15c200 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.692+0000 7ff1412a9640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff12806c920 0x7ff12806ede0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.692+0000 7ff1412a9640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c1f9010 0x7ff13c1ea3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.692+0000 7ff1412a9640 1 --2- 172.21.15.37:0/2623596870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff124004800 0x7ff13c0cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.692+0000 7ff1412a9640 1 -- 172.21.15.37:0/2623596870 >> 172.21.15.37:0/2623596870 conn(0x7ff13c0d5160 msgr2=0x7ff13c06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:32.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.693+0000 7ff1412a9640 1 -- 172.21.15.37:0/2623596870 shutdown_connections 2024-04-26T20:22:32.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.693+0000 7ff1412a9640 1 -- 172.21.15.37:0/2623596870 wait complete. 2024-04-26T20:22:32.707 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:22:32.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.857+0000 7f13ab371640 1 Processor -- start 2024-04-26T20:22:32.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.857+0000 7f13ab371640 1 -- start start 2024-04-26T20:22:32.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13ab371640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f13a415f350 0x7f13a40ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:32.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13ab371640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f13a40683f0 0x7f13a406ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:32.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13ab371640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a4070720 con 0x7f13a406d3c0 2024-04-26T20:22:32.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13ab371640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f13a415f730 con 0x7f13a415f350 2024-04-26T20:22:32.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13ab371640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f13a4163de0 con 0x7f13a40683f0 2024-04-26T20:22:32.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13a910d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f13a415f350 0x7f13a40ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:32.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13a910d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f13a415f350 0x7f13a40ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58152/0 (socket says 172.21.15.37:58152) 2024-04-26T20:22:32.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13a910d640 1 -- 172.21.15.37:0/1764912226 learned_addr learned my addr 172.21.15.37:0/1764912226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:32.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13a890c640 1 --2- 172.21.15.37:0/1764912226 >> v2:172.21.15.134:3300/0 conn(0x7f13a40683f0 0x7f13a406ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:32.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13a910d640 1 -- 172.21.15.37:0/1764912226 >> v2:172.21.15.134:3300/0 conn(0x7f13a40683f0 msgr2=0x7f13a406ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:32.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13a910d640 1 --2- 172.21.15.37:0/1764912226 >> v2:172.21.15.134:3300/0 conn(0x7f13a40683f0 0x7f13a406ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13a910d640 1 -- 172.21.15.37:0/1764912226 >> v1:172.21.15.134:6789/0 conn(0x7f13a406d3c0 legacy=0x7f13a40b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:32.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13a910d640 1 -- 172.21.15.37:0/1764912226 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a406c3e0 con 0x7f13a415f350 2024-04-26T20:22:32.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.858+0000 7f13a890c640 1 --2- 172.21.15.37:0/1764912226 >> v2:172.21.15.134:3300/0 conn(0x7f13a40683f0 0x7f13a406ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:32.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.859+0000 7f13a910d640 1 --2- 172.21.15.37:0/1764912226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a415f350 0x7f13a40ae8c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f13a40cdd40 tx=0x7f139c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af88cd953ec16d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:32.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.859+0000 7f13a3fff640 1 -- 172.21.15.37:0/1764912226 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c037070 con 0x7f13a415f350 2024-04-26T20:22:32.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.859+0000 7f13a3fff640 1 -- 172.21.15.37:0/1764912226 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c032440 con 0x7f13a415f350 2024-04-26T20:22:32.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.859+0000 7f13a3fff640 1 -- 172.21.15.37:0/1764912226 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c032810 con 0x7f13a415f350 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.860+0000 7f13ab371640 1 -- 172.21.15.37:0/1764912226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a415f350 msgr2=0x7f13a40ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.860+0000 7f13ab371640 1 --2- 172.21.15.37:0/1764912226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a415f350 0x7f13a40ae8c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f13a40cdd40 tx=0x7f139c02a230 comp rx=0 tx=0).stop 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.860+0000 7f13ab371640 1 -- 172.21.15.37:0/1764912226 shutdown_connections 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.860+0000 7f13ab371640 1 --2- 172.21.15.37:0/1764912226 >> v2:172.21.15.134:3300/0 conn(0x7f13a40683f0 0x7f13a406ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.860+0000 7f13ab371640 1 --2- 172.21.15.37:0/1764912226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a415f350 0x7f13a40ae8c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.860+0000 7f13ab371640 1 -- 172.21.15.37:0/1764912226 >> 172.21.15.37:0/1764912226 conn(0x7f13a40d15c0 msgr2=0x7f13a40d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.860+0000 7f13ab371640 1 -- 172.21.15.37:0/1764912226 shutdown_connections 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.860+0000 7f13ab371640 1 -- 172.21.15.37:0/1764912226 wait complete. 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.861+0000 7f13ab371640 1 Processor -- start 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.861+0000 7f13ab371640 1 -- start start 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13ab371640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a40683f0 0x7f13a406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:32.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13ab371640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a406d3c0 0x7f13a41604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13ab371640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a415f350 0x7f13a4154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13ab371640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4163b00 con 0x7f13a406d3c0 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13ab371640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13a406c9f0 con 0x7f13a415f350 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13ab371640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13a406c680 con 0x7f13a40683f0 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13a890c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a406d3c0 0x7f13a41604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13a890c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a406d3c0 0x7f13a41604c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58160/0 (socket says 172.21.15.37:58160) 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13a890c640 1 -- 172.21.15.37:0/4044669583 learned_addr learned my addr 172.21.15.37:0/4044669583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13a910d640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a40683f0 0x7f13a406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13a990e640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a415f350 0x7f13a4154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13a890c640 1 -- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a40683f0 msgr2=0x7f13a406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13a890c640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a40683f0 0x7f13a406fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13a890c640 1 -- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a415f350 msgr2=0x7f13a4154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13a890c640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a415f350 0x7f13a4154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.862+0000 7f13a890c640 1 -- 172.21.15.37:0/4044669583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a41598a0 con 0x7f13a406d3c0 2024-04-26T20:22:32.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.863+0000 7f13a890c640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a406d3c0 0x7f13a41604c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f139800c7f0 tx=0x7f139800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:32.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.863+0000 7f13a1ffb640 1 -- 172.21.15.37:0/4044669583 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398015070 con 0x7f13a406d3c0 2024-04-26T20:22:32.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.863+0000 7f13a1ffb640 1 -- 172.21.15.37:0/4044669583 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1398007a30 con 0x7f13a406d3c0 2024-04-26T20:22:32.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.863+0000 7f13ab371640 1 -- 172.21.15.37:0/4044669583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a41595c0 con 0x7f13a406d3c0 2024-04-26T20:22:32.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.863+0000 7f13a1ffb640 1 -- 172.21.15.37:0/4044669583 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398007e00 con 0x7f13a406d3c0 2024-04-26T20:22:32.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.863+0000 7f13ab371640 1 -- 172.21.15.37:0/4044669583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a415ffa0 con 0x7f13a406d3c0 2024-04-26T20:22:32.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.864+0000 7f13ab371640 1 -- 172.21.15.37:0/4044669583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a406a360 con 0x7f13a406d3c0 2024-04-26T20:22:32.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.864+0000 7f13a1ffb640 1 -- 172.21.15.37:0/4044669583 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1398018070 con 0x7f13a406d3c0 2024-04-26T20:22:32.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.865+0000 7f13a1ffb640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f139006c8f0 0x7f139006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:32.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.865+0000 7f13a1ffb640 1 -- 172.21.15.37:0/4044669583 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f139808bb50 con 0x7f13a406d3c0 2024-04-26T20:22:32.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.865+0000 7f13a910d640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f139006c8f0 0x7f139006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:32.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.866+0000 7f13a910d640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f139006c8f0 0x7f139006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f139c034000 tx=0x7f139c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:32.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:32.869+0000 7f13a1ffb640 1 -- 172.21.15.37:0/4044669583 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1398059f40 con 0x7f13a406d3c0 2024-04-26T20:22:33.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.053+0000 7f13ab371640 1 -- 172.21.15.37:0/4044669583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f13a4160170 con 0x7f13a406d3c0 2024-04-26T20:22:33.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.691+0000 7f13a1ffb640 1 -- 172.21.15.37:0/4044669583 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f1398059840 con 0x7f13a406d3c0 2024-04-26T20:22:33.693 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:22:33.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 -- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f139006c8f0 msgr2=0x7f139006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:33.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f139006c8f0 0x7f139006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f139c034000 tx=0x7f139c02e000 comp rx=0 tx=0).stop 2024-04-26T20:22:33.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 -- 172.21.15.37:0/4044669583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a406d3c0 msgr2=0x7f13a41604c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:33.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a406d3c0 0x7f13a41604c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f139800c7f0 tx=0x7f139800cbf0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 -- 172.21.15.37:0/4044669583 shutdown_connections 2024-04-26T20:22:33.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a415f350 0x7f13a4154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a406d3c0 0x7f13a41604c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f139006c8f0 0x7f139006edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 --2- 172.21.15.37:0/4044669583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a40683f0 0x7f13a406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 -- 172.21.15.37:0/4044669583 >> 172.21.15.37:0/4044669583 conn(0x7f13a40d15c0 msgr2=0x7f13a40b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:33.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 -- 172.21.15.37:0/4044669583 shutdown_connections 2024-04-26T20:22:33.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.695+0000 7f13ab371640 1 -- 172.21.15.37:0/4044669583 wait complete. 2024-04-26T20:22:33.711 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:22:33.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.820+0000 7f4abc4dc640 1 Processor -- start 2024-04-26T20:22:33.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.820+0000 7f4abc4dc640 1 -- start start 2024-04-26T20:22:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.821+0000 7f4abc4dc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ab415f410 0x7f4ab40b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.821+0000 7f4abc4dc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4ab40d3e80 0x7f4ab40d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.821+0000 7f4abc4dc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ab40b55a0 con 0x7f4ab415f7f0 2024-04-26T20:22:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.821+0000 7f4abc4dc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4ab40b5880 con 0x7f4ab40d3e80 2024-04-26T20:22:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.821+0000 7f4abc4dc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab40b5b60 con 0x7f4ab415f410 2024-04-26T20:22:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.821+0000 7f4abaa79640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4ab40d3e80 0x7f4ab40d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.821+0000 7f4ab9a77640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4ab415f7f0 0x7f4ab40d3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57382/0 (socket says 172.21.15.37:57382) 2024-04-26T20:22:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.821+0000 7f4ab9a77640 1 -- 172.21.15.37:0/3540640786 learned_addr learned my addr 172.21.15.37:0/3540640786 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.822+0000 7f4aba278640 1 --2- 172.21.15.37:0/3540640786 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ab415f410 0x7f4ab40b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.822+0000 7f4abaa79640 1 -- 172.21.15.37:0/3540640786 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ab415f410 msgr2=0x7f4ab40b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.822+0000 7f4abaa79640 1 --2- 172.21.15.37:0/3540640786 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ab415f410 0x7f4ab40b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.822+0000 7f4abaa79640 1 -- 172.21.15.37:0/3540640786 >> v1:172.21.15.37:6789/0 conn(0x7f4ab415f7f0 legacy=0x7f4ab40d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.822+0000 7f4ab9276640 1 -- 172.21.15.37:0/3540640786 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2961859998 0 0) 0x7f4ab40b55a0 con 0x7f4ab415f7f0 2024-04-26T20:22:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.822+0000 7f4abaa79640 1 -- 172.21.15.37:0/3540640786 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab40b5e40 con 0x7f4ab40d3e80 2024-04-26T20:22:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.822+0000 7f4abaa79640 1 --2- 172.21.15.37:0/3540640786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab40d3e80 0x7f4ab40d0110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009d90 tx=0x7f4aa402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f52775b411960c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.823+0000 7f4ab9276640 1 -- 172.21.15.37:0/3540640786 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4007740 con 0x7f4ab40d3e80 2024-04-26T20:22:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.823+0000 7f4ab9276640 1 -- 172.21.15.37:0/3540640786 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa40078e0 con 0x7f4ab40d3e80 2024-04-26T20:22:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.823+0000 7f4ab9276640 1 -- 172.21.15.37:0/3540640786 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4007cb0 con 0x7f4ab40d3e80 2024-04-26T20:22:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.823+0000 7f4abc4dc640 1 -- 172.21.15.37:0/3540640786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab40d3e80 msgr2=0x7f4ab40d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.823+0000 7f4abc4dc640 1 --2- 172.21.15.37:0/3540640786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab40d3e80 0x7f4ab40d0110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009d90 tx=0x7f4aa402a6f0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.823+0000 7f4abc4dc640 1 -- 172.21.15.37:0/3540640786 shutdown_connections 2024-04-26T20:22:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.823+0000 7f4abc4dc640 1 --2- 172.21.15.37:0/3540640786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab40d3e80 0x7f4ab40d0110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.823+0000 7f4abc4dc640 1 --2- 172.21.15.37:0/3540640786 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ab415f410 0x7f4ab40b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.823+0000 7f4abc4dc640 1 -- 172.21.15.37:0/3540640786 >> 172.21.15.37:0/3540640786 conn(0x7f4ab408ebb0 msgr2=0x7f4ab409e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.823+0000 7f4abc4dc640 1 -- 172.21.15.37:0/3540640786 shutdown_connections 2024-04-26T20:22:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.823+0000 7f4abc4dc640 1 -- 172.21.15.37:0/3540640786 wait complete. 2024-04-26T20:22:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.824+0000 7f4abc4dc640 1 Processor -- start 2024-04-26T20:22:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.824+0000 7f4abc4dc640 1 -- start start 2024-04-26T20:22:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.825+0000 7f4abc4dc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ab40d3e80 0x7f4ab406d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.825+0000 7f4abc4dc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ab415f410 0x7f4ab40c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.825+0000 7f4abc4dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab40ba570 0x7f4ab40caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.825+0000 7f4abc4dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab40b5240 con 0x7f4ab40ba570 2024-04-26T20:22:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.825+0000 7f4abc4dc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab40b50c0 con 0x7f4ab415f410 2024-04-26T20:22:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.825+0000 7f4abc4dc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab40d0650 con 0x7f4ab40d3e80 2024-04-26T20:22:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.825+0000 7f4abaa79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab40ba570 0x7f4ab40caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.825+0000 7f4abaa79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab40ba570 0x7f4ab40caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58180/0 (socket says 172.21.15.37:58180) 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.825+0000 7f4abaa79640 1 -- 172.21.15.37:0/1177520828 learned_addr learned my addr 172.21.15.37:0/1177520828 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.825+0000 7f4aba278640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ab40d3e80 0x7f4ab406d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.825+0000 7f4ab9a77640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ab415f410 0x7f4ab40c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.826+0000 7f4abaa79640 1 -- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ab40d3e80 msgr2=0x7f4ab406d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.826+0000 7f4abaa79640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ab40d3e80 0x7f4ab406d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.826+0000 7f4abaa79640 1 -- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ab415f410 msgr2=0x7f4ab40c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.826+0000 7f4abaa79640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ab415f410 0x7f4ab40c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.826+0000 7f4abaa79640 1 -- 172.21.15.37:0/1177520828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab406e7e0 con 0x7f4ab40ba570 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.826+0000 7f4abaa79640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab40ba570 0x7f4ab40caeb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009d90 tx=0x7f4aa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.826+0000 7f4ab2ffd640 1 -- 172.21.15.37:0/1177520828 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4040070 con 0x7f4ab40ba570 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.826+0000 7f4ab2ffd640 1 -- 172.21.15.37:0/1177520828 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa4031c40 con 0x7f4ab40ba570 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.826+0000 7f4abc4dc640 1 -- 172.21.15.37:0/1177520828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab406df40 con 0x7f4ab40ba570 2024-04-26T20:22:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.826+0000 7f4ab2ffd640 1 -- 172.21.15.37:0/1177520828 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4009450 con 0x7f4ab40ba570 2024-04-26T20:22:33.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.826+0000 7f4abc4dc640 1 -- 172.21.15.37:0/1177520828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab406e500 con 0x7f4ab40ba570 2024-04-26T20:22:33.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.827+0000 7f4abc4dc640 1 -- 172.21.15.37:0/1177520828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab406a360 con 0x7f4ab40ba570 2024-04-26T20:22:33.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.828+0000 7f4ab2ffd640 1 -- 172.21.15.37:0/1177520828 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4aa4039040 con 0x7f4ab40ba570 2024-04-26T20:22:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.828+0000 7f4ab2ffd640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4aa006c8f0 0x7f4aa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.828+0000 7f4ab2ffd640 1 -- 172.21.15.37:0/1177520828 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7f4aa40ac830 con 0x7f4ab40ba570 2024-04-26T20:22:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.828+0000 7f4aba278640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4aa006c8f0 0x7f4aa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.829+0000 7f4aba278640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4aa006c8f0 0x7f4aa006edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4aac006fd0 tx=0x7f4aac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:33.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:33.832+0000 7f4ab2ffd640 1 -- 172.21.15.37:0/1177520828 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4aa407a780 con 0x7f4ab40ba570 2024-04-26T20:22:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.027+0000 7f4abc4dc640 1 -- 172.21.15.37:0/1177520828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4ab40c99f0 con 0x7f4ab40ba570 2024-04-26T20:22:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.693+0000 7f4ab2ffd640 1 -- 172.21.15.37:0/1177520828 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f4aa407ea80 con 0x7f4ab40ba570 2024-04-26T20:22:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-26T20:22:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 -- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4aa006c8f0 msgr2=0x7f4aa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4aa006c8f0 0x7f4aa006edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4aac006fd0 tx=0x7f4aac008040 comp rx=0 tx=0).stop 2024-04-26T20:22:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 -- 172.21.15.37:0/1177520828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab40ba570 msgr2=0x7f4ab40caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab40ba570 0x7f4ab40caeb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009d90 tx=0x7f4aa4004a40 comp rx=0 tx=0).stop 2024-04-26T20:22:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 -- 172.21.15.37:0/1177520828 shutdown_connections 2024-04-26T20:22:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab40ba570 0x7f4ab40caeb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ab415f410 0x7f4ab40c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4aa006c8f0 0x7f4aa006edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 --2- 172.21.15.37:0/1177520828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ab40d3e80 0x7f4ab406d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 -- 172.21.15.37:0/1177520828 >> 172.21.15.37:0/1177520828 conn(0x7f4ab408ebb0 msgr2=0x7f4ab415e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 -- 172.21.15.37:0/1177520828 shutdown_connections 2024-04-26T20:22:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.696+0000 7f4abc4dc640 1 -- 172.21.15.37:0/1177520828 wait complete. 2024-04-26T20:22:34.714 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:22:34.714 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T20:22:34.715 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:22:34.715 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:22:34.715 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:22:34.715 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T20:22:34.715 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:22:34.715 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:22:34.715 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:22:34.716 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T20:22:34.716 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:22:34.716 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:22:34.716 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:22:34.716 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T20:22:34.716 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:22:34.716 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:22:34.726 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:22:34.726 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:22:34.727 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:22:34.727 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:22:34.728 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:22:34.728 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:22:34.729 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:22:34.729 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:22:34.768 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:22:34.769 DEBUG:teuthology.parallel:result is None 2024-04-26T20:22:34.769 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:22:34.769 DEBUG:teuthology.parallel:result is None 2024-04-26T20:22:34.771 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:22:34.771 DEBUG:teuthology.parallel:result is None 2024-04-26T20:22:34.772 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:22:34.772 DEBUG:teuthology.parallel:result is None 2024-04-26T20:22:34.772 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T20:22:34.851 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-26T20:22:34.851 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-26T20:22:34.852 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-04-26T20:22:34.853 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-04-26T20:22:34.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.882+0000 7fb28d1e2640 1 Processor -- start 2024-04-26T20:22:34.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.882+0000 7fb28d1e2640 1 -- start start 2024-04-26T20:22:34.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.883+0000 7fb28d1e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2881609a0 0x7fb2880cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:34.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.883+0000 7fb28d1e2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb2880d2290 0x7fb2880ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:34.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.883+0000 7fb28d1e2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb28806e5a0 con 0x7fb2880cfe70 2024-04-26T20:22:34.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.883+0000 7fb28d1e2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb28806fc40 con 0x7fb2880d2290 2024-04-26T20:22:34.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.883+0000 7fb28d1e2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb2880d02b0 con 0x7fb2881609a0 2024-04-26T20:22:34.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.883+0000 7fb28759e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb2880d2290 0x7fb2880ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:34.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.883+0000 7fb28659c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb2880cfe70 0x7fb2880cdf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:41148/0 (socket says 172.21.15.37:41148) 2024-04-26T20:22:34.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.883+0000 7fb28659c640 1 -- 172.21.15.37:0/2614080267 learned_addr learned my addr 172.21.15.37:0/2614080267 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:34.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.883+0000 7fb286d9d640 1 --2- 172.21.15.37:0/2614080267 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2881609a0 0x7fb2880cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:34.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.883+0000 7fb285d9b640 1 -- 172.21.15.37:0/2614080267 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532401463 0 0) 0x7fb28806e5a0 con 0x7fb2880cfe70 2024-04-26T20:22:34.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.884+0000 7fb286d9d640 1 -- 172.21.15.37:0/2614080267 >> v2:172.21.15.134:3300/0 conn(0x7fb2880d2290 msgr2=0x7fb2880ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:34.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.884+0000 7fb286d9d640 1 --2- 172.21.15.37:0/2614080267 >> v2:172.21.15.134:3300/0 conn(0x7fb2880d2290 0x7fb2880ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.884+0000 7fb286d9d640 1 -- 172.21.15.37:0/2614080267 >> v1:172.21.15.134:6789/0 conn(0x7fb2880cfe70 legacy=0x7fb2880cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:34.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.884+0000 7fb286d9d640 1 -- 172.21.15.37:0/2614080267 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2880cd720 con 0x7fb2881609a0 2024-04-26T20:22:34.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.884+0000 7fb28759e640 1 --2- 172.21.15.37:0/2614080267 >> v2:172.21.15.134:3300/0 conn(0x7fb2880d2290 0x7fb2880ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:34.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.884+0000 7fb286d9d640 1 --2- 172.21.15.37:0/2614080267 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2881609a0 0x7fb2880cf930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2780097a0 tx=0x7fb27802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=813e1472206a8f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:34.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.885+0000 7fb285d9b640 1 -- 172.21.15.37:0/2614080267 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278037070 con 0x7fb2881609a0 2024-04-26T20:22:34.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.885+0000 7fb285d9b640 1 -- 172.21.15.37:0/2614080267 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb278032440 con 0x7fb2881609a0 2024-04-26T20:22:34.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.885+0000 7fb285d9b640 1 -- 172.21.15.37:0/2614080267 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278032810 con 0x7fb2881609a0 2024-04-26T20:22:34.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.885+0000 7fb28d1e2640 1 -- 172.21.15.37:0/2614080267 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2881609a0 msgr2=0x7fb2880cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:34.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.885+0000 7fb28d1e2640 1 --2- 172.21.15.37:0/2614080267 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2881609a0 0x7fb2880cf930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2780097a0 tx=0x7fb27802a180 comp rx=0 tx=0).stop 2024-04-26T20:22:34.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.885+0000 7fb28d1e2640 1 -- 172.21.15.37:0/2614080267 shutdown_connections 2024-04-26T20:22:34.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.885+0000 7fb28d1e2640 1 --2- 172.21.15.37:0/2614080267 >> v2:172.21.15.134:3300/0 conn(0x7fb2880d2290 0x7fb2880ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.885+0000 7fb28d1e2640 1 --2- 172.21.15.37:0/2614080267 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2881609a0 0x7fb2880cf930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.885+0000 7fb28d1e2640 1 -- 172.21.15.37:0/2614080267 >> 172.21.15.37:0/2614080267 conn(0x7fb28809ba50 msgr2=0x7fb28809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:34.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.885+0000 7fb28d1e2640 1 -- 172.21.15.37:0/2614080267 shutdown_connections 2024-04-26T20:22:34.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.885+0000 7fb28d1e2640 1 -- 172.21.15.37:0/2614080267 wait complete. 2024-04-26T20:22:34.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.887+0000 7fb28d1e2640 1 Processor -- start 2024-04-26T20:22:34.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.887+0000 7fb28d1e2640 1 -- start start 2024-04-26T20:22:34.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.887+0000 7fb28d1e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2880cfe70 0x7fb288159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:34.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.887+0000 7fb28d1e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2880d2290 0x7fb288159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:34.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.887+0000 7fb28d1e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2881609a0 0x7fb2880a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:34.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.887+0000 7fb28d1e2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb28806d350 con 0x7fb2880d2290 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.887+0000 7fb28d1e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb28806e880 con 0x7fb2881609a0 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.887+0000 7fb28d1e2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb28806d4d0 con 0x7fb2880cfe70 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2880d2290 0x7fb288159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2880d2290 0x7fb288159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58216/0 (socket says 172.21.15.37:58216) 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28659c640 1 -- 172.21.15.37:0/216844815 learned_addr learned my addr 172.21.15.37:0/216844815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb286d9d640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2880cfe70 0x7fb288159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28759e640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2881609a0 0x7fb2880a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28659c640 1 -- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2880cfe70 msgr2=0x7fb288159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28659c640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2880cfe70 0x7fb288159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28659c640 1 -- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2881609a0 msgr2=0x7fb2880a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28659c640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2881609a0 0x7fb2880a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28659c640 1 -- 172.21.15.37:0/216844815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2881f5b00 con 0x7fb2880d2290 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28759e640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2881609a0 0x7fb2880a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb286d9d640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2880cfe70 0x7fb288159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:34.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28659c640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2880d2290 0x7fb288159090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00c430 tx=0x7fb27c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:34.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb26f7fe640 1 -- 172.21.15.37:0/216844815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0074e0 con 0x7fb2880d2290 2024-04-26T20:22:34.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb26f7fe640 1 -- 172.21.15.37:0/216844815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27c0047e0 con 0x7fb2880d2290 2024-04-26T20:22:34.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28d1e2640 1 -- 172.21.15.37:0/216844815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2881f5de0 con 0x7fb2880d2290 2024-04-26T20:22:34.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb26f7fe640 1 -- 172.21.15.37:0/216844815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c004bd0 con 0x7fb2880d2290 2024-04-26T20:22:34.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.888+0000 7fb28d1e2640 1 -- 172.21.15.37:0/216844815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2881f63a0 con 0x7fb2880d2290 2024-04-26T20:22:34.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.889+0000 7fb28d1e2640 1 -- 172.21.15.37:0/216844815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2880693e0 con 0x7fb2880d2290 2024-04-26T20:22:34.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.890+0000 7fb26f7fe640 1 -- 172.21.15.37:0/216844815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb27c011070 con 0x7fb2880d2290 2024-04-26T20:22:34.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.890+0000 7fb26f7fe640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb27406c8f0 0x7fb27406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:34.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.890+0000 7fb26f7fe640 1 -- 172.21.15.37:0/216844815 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7fb27c08b5d0 con 0x7fb2880d2290 2024-04-26T20:22:34.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.890+0000 7fb286d9d640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb27406c8f0 0x7fb27406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:34.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.891+0000 7fb286d9d640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb27406c8f0 0x7fb27406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb278034000 tx=0x7fb27802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:34.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:34.895+0000 7fb26f7fe640 1 -- 172.21.15.37:0/216844815 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb27c059c60 con 0x7fb2880d2290 2024-04-26T20:22:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.095+0000 7fb28d1e2640 1 -- 172.21.15.37:0/216844815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb2880d2a60 con 0x7fb2880d2290 2024-04-26T20:22:35.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.710+0000 7fb26f7fe640 1 -- 172.21.15.37:0/216844815 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7fb27c059560 con 0x7fb2880d2290 2024-04-26T20:22:35.712 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-04-26T20:22:35.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.713+0000 7fb28d1e2640 1 -- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb27406c8f0 msgr2=0x7fb27406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:35.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.713+0000 7fb28d1e2640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb27406c8f0 0x7fb27406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb278034000 tx=0x7fb27802e000 comp rx=0 tx=0).stop 2024-04-26T20:22:35.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.713+0000 7fb28d1e2640 1 -- 172.21.15.37:0/216844815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2880d2290 msgr2=0x7fb288159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:35.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.713+0000 7fb28d1e2640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2880d2290 0x7fb288159090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00c430 tx=0x7fb27c00c8f0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.713+0000 7fb28d1e2640 1 -- 172.21.15.37:0/216844815 shutdown_connections 2024-04-26T20:22:35.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.713+0000 7fb28d1e2640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2881609a0 0x7fb2880a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.713+0000 7fb28d1e2640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2880d2290 0x7fb288159090 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.713+0000 7fb28d1e2640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb27406c8f0 0x7fb27406edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.713+0000 7fb28d1e2640 1 --2- 172.21.15.37:0/216844815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2880cfe70 0x7fb288159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.713+0000 7fb28d1e2640 1 -- 172.21.15.37:0/216844815 >> 172.21.15.37:0/216844815 conn(0x7fb28809ba50 msgr2=0x7fb2880b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:35.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.714+0000 7fb28d1e2640 1 -- 172.21.15.37:0/216844815 shutdown_connections 2024-04-26T20:22:35.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.714+0000 7fb28d1e2640 1 -- 172.21.15.37:0/216844815 wait complete. 2024-04-26T20:22:35.729 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T20:22:35.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.833+0000 7f1edd7f7640 1 Processor -- start 2024-04-26T20:22:35.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.834+0000 7f1edd7f7640 1 -- start start 2024-04-26T20:22:35.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.834+0000 7f1edd7f7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1ed8162b40 0x7f1ed80d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:35.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.834+0000 7f1edd7f7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ed80d3c80 0x7f1ed80d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:35.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.834+0000 7f1edd7f7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed80d0210 con 0x7f1ed80d3820 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.834+0000 7f1edd7f7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed80d04f0 con 0x7f1ed80d3c80 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.834+0000 7f1edd7f7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed80d07d0 con 0x7f1ed8162b40 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.834+0000 7f1ed77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ed80d3c80 0x7f1ed80d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.835+0000 7f1ed77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ed80d3c80 0x7f1ed80d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48152/0 (socket says 172.21.15.37:48152) 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.835+0000 7f1ed77fe640 1 -- 172.21.15.37:0/3397722655 learned_addr learned my addr 172.21.15.37:0/3397722655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.835+0000 7f1ed6ffd640 1 --2- 172.21.15.37:0/3397722655 >> v2:172.21.15.134:3300/0 conn(0x7f1ed8162b40 0x7f1ed80d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.835+0000 7f1ed77fe640 1 -- 172.21.15.37:0/3397722655 >> v2:172.21.15.134:3300/0 conn(0x7f1ed8162b40 msgr2=0x7f1ed80d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.835+0000 7f1ed77fe640 1 --2- 172.21.15.37:0/3397722655 >> v2:172.21.15.134:3300/0 conn(0x7f1ed8162b40 0x7f1ed80d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.835+0000 7f1ed77fe640 1 -- 172.21.15.37:0/3397722655 >> v1:172.21.15.134:6789/0 conn(0x7f1ed80d3820 legacy=0x7f1ed80d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.835+0000 7f1ed77fe640 1 -- 172.21.15.37:0/3397722655 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed80d0ab0 con 0x7f1ed80d3c80 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.835+0000 7f1ed6ffd640 1 --2- 172.21.15.37:0/3397722655 >> v2:172.21.15.134:3300/0 conn(0x7f1ed8162b40 0x7f1ed80d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:35.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1ed77fe640 1 --2- 172.21.15.37:0/3397722655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80d3c80 0x7f1ed80d1f60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80d2910 tx=0x7f1ed002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e899207797ed01d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:35.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1ed67fc640 1 -- 172.21.15.37:0/3397722655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0036070 con 0x7f1ed80d3c80 2024-04-26T20:22:35.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1ed67fc640 1 -- 172.21.15.37:0/3397722655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed0031600 con 0x7f1ed80d3c80 2024-04-26T20:22:35.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1ed67fc640 1 -- 172.21.15.37:0/3397722655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00319d0 con 0x7f1ed80d3c80 2024-04-26T20:22:35.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1edd7f7640 1 -- 172.21.15.37:0/3397722655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80d3c80 msgr2=0x7f1ed80d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:35.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1edd7f7640 1 --2- 172.21.15.37:0/3397722655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80d3c80 0x7f1ed80d1f60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80d2910 tx=0x7f1ed002a780 comp rx=0 tx=0).stop 2024-04-26T20:22:35.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1edd7f7640 1 -- 172.21.15.37:0/3397722655 shutdown_connections 2024-04-26T20:22:35.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1edd7f7640 1 --2- 172.21.15.37:0/3397722655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80d3c80 0x7f1ed80d1f60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1edd7f7640 1 --2- 172.21.15.37:0/3397722655 >> v2:172.21.15.134:3300/0 conn(0x7f1ed8162b40 0x7f1ed80d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1edd7f7640 1 -- 172.21.15.37:0/3397722655 >> 172.21.15.37:0/3397722655 conn(0x7f1ed809e550 msgr2=0x7f1ed80d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:35.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1edd7f7640 1 -- 172.21.15.37:0/3397722655 shutdown_connections 2024-04-26T20:22:35.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.836+0000 7f1edd7f7640 1 -- 172.21.15.37:0/3397722655 wait complete. 2024-04-26T20:22:35.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.837+0000 7f1edd7f7640 1 Processor -- start 2024-04-26T20:22:35.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.838+0000 7f1edd7f7640 1 -- start start 2024-04-26T20:22:35.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.838+0000 7f1edd7f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80d3820 0x7f1ed80ba7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:35.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.838+0000 7f1edd7f7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ed80d3c80 0x7f1ed814d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:35.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.838+0000 7f1edd7f7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ed8162b40 0x7f1ed80cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:35.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.838+0000 7f1edd7f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed80b5bb0 con 0x7f1ed80d3820 2024-04-26T20:22:35.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.838+0000 7f1edd7f7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed80d4110 con 0x7f1ed80d3c80 2024-04-26T20:22:35.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.838+0000 7f1edd7f7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed80d6110 con 0x7f1ed8162b40 2024-04-26T20:22:35.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.838+0000 7f1ed6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80d3820 0x7f1ed80ba7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.838+0000 7f1ed6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80d3820 0x7f1ed80ba7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48154/0 (socket says 172.21.15.37:48154) 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.838+0000 7f1ed6ffd640 1 -- 172.21.15.37:0/4138215575 learned_addr learned my addr 172.21.15.37:0/4138215575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1ece7fc640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ed80d3c80 0x7f1ed814d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1ed77fe640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ed8162b40 0x7f1ed80cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1ed6ffd640 1 -- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ed8162b40 msgr2=0x7f1ed80cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1ed6ffd640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ed8162b40 0x7f1ed80cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1ed6ffd640 1 -- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ed80d3c80 msgr2=0x7f1ed814d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1ed6ffd640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ed80d3c80 0x7f1ed814d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1ed6ffd640 1 -- 172.21.15.37:0/4138215575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed80b27a0 con 0x7f1ed80d3820 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1ed6ffd640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80d3820 0x7f1ed80ba7d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800c7f0 tx=0x7f1ec800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1ecffff640 1 -- 172.21.15.37:0/4138215575 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8015070 con 0x7f1ed80d3820 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1ecffff640 1 -- 172.21.15.37:0/4138215575 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec8007a30 con 0x7f1ed80d3820 2024-04-26T20:22:35.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1edd7f7640 1 -- 172.21.15.37:0/4138215575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed80b2a80 con 0x7f1ed80d3820 2024-04-26T20:22:35.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1ecffff640 1 -- 172.21.15.37:0/4138215575 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8007e00 con 0x7f1ed80d3820 2024-04-26T20:22:35.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.839+0000 7f1edd7f7640 1 -- 172.21.15.37:0/4138215575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed80b3040 con 0x7f1ed80d3820 2024-04-26T20:22:35.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.840+0000 7f1edd7f7640 1 -- 172.21.15.37:0/4138215575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed806a360 con 0x7f1ed80d3820 2024-04-26T20:22:35.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.841+0000 7f1ecffff640 1 -- 172.21.15.37:0/4138215575 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ec8018070 con 0x7f1ed80d3820 2024-04-26T20:22:35.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.841+0000 7f1ecffff640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb806c8f0 0x7f1eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:35.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.841+0000 7f1ecffff640 1 -- 172.21.15.37:0/4138215575 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f1ec808b710 con 0x7f1ed80d3820 2024-04-26T20:22:35.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.841+0000 7f1ece7fc640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb806c8f0 0x7f1eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:35.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.842+0000 7f1ece7fc640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb806c8f0 0x7f1eb806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ed806dd60 tx=0x7f1ec0009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:35.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:35.845+0000 7f1ecffff640 1 -- 172.21.15.37:0/4138215575 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ec8059840 con 0x7f1ed80d3820 2024-04-26T20:22:36.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.104+0000 7f1edd7f7640 1 -- 172.21.15.37:0/4138215575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1ed80d2960 con 0x7f1ed80d3820 2024-04-26T20:22:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.105+0000 7f1ecffff640 1 -- 172.21.15.37:0/4138215575 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f1ec805db40 con 0x7f1ed80d3820 2024-04-26T20:22:36.106 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:36.106 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:22:36.106 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-04-26T20:22:36.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.106 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-04-26T20:22:36.106 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxmMZ1xLRAA25Lk5iPjTAlo8+3CnWPavg==", 2024-04-26T20:22:36.106 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxmSCb5OBAA1qn4Xl59ur/wJDapWSLZ9A==", 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-26T20:22:36.107 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxmxYZlNBAAqm/foyZ1j157Yi/vkOspOg==", 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.108 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCkDCxmx942BBAAJyAAYb/ZMcI0ZYLluqttZw==", 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCnDCxmuavuHhAAHPS8O9ExPpvdHT10wJDHrw==", 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:22:36.109 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.110 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.110 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-04-26T20:22:36.110 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCqDCxmHDa3HRAAVysGtbYHxPLpHx3kdS1EKw==", 2024-04-26T20:22:36.110 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.110 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:22:36.110 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCtDCxmwplYHBAAxzI1MLVpn8kK1B27n/ZBoA==", 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.112 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-04-26T20:22:36.112 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCwDCxmlem9GxAAR6h6rw0owBPPUT/Eyh26bQ==", 2024-04-26T20:22:36.112 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.112 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:22:36.112 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:22:36.112 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.112 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.113 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-04-26T20:22:36.113 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC2DCxmqKnHHhAAkvIA0VP38Y2RYBKMdZFuZA==", 2024-04-26T20:22:36.113 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.113 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:22:36.113 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:22:36.113 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.113 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.114 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-04-26T20:22:36.114 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC5DCxmGU7hHBAAAXOqhBDJW/ZiUYlY4dv/DA==", 2024-04-26T20:22:36.114 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.114 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:22:36.114 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:22:36.114 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.114 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.114 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC8DCxmF7uHGhAAKioy1jI7alaj7WSVHJ1nKA==", 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC/DCxmKk8wGRAAaghSXnN4nUlxzhkYGBzZig==", 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:22:36.115 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCkDCxmQTcnCBAAFh1Ew2ihHFtREYikzwF+Dw==", 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-04-26T20:22:36.116 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCkDCxmXGS+EBAAvWfY/Xt9yfBLUr3jhc44cA==", 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCkDCxmUdq2DRAAq/Hml2I/z0bkKXjN9jIeDg==", 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-26T20:22:36.117 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCkDCxmtrYnFhAADmyeYHIoG22nzUEozY0R9Q==", 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.118 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxmazwnERAAfhEVfmWlGnPrPPmMWxbOKQ==", 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-mds", 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQDFDCxmcgUVCxAA4mrYswtM9QXonkEWHqED1w==", 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQDFDCxmgw0VCxAAzbhiZa0U9rsRg0QOnoqTYA==", 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-osd", 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQDFDCxm4hUVCxAA7iTw/UR8rLiutsvBqmM/Mg==", 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQDFDCxm6hwVCxAAoJM94l5hDTZ4L8mye5TW9w==", 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQDFDCxmMSQVCxAAQ3nZ5SCipX56kKJ6Sj6lWQ==", 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.121 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQDFDCxmuSsVCxAALrWlOZxWUVCzDTNPNZRrQw==", 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxm5VtOKhAA9ZJhTDAGuUn+Un63aSPDbQ==", 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:22:36.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:36.123 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-04-26T20:22:36.123 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxmUhBnJRAAGsrfntQyYUoHJlj2M2OCVA==", 2024-04-26T20:22:36.123 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:22:36.123 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-26T20:22:36.123 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-26T20:22:36.123 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:22:36.123 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.123 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:36.123 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T20:22:36.123 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:22:36.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 -- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb806c8f0 msgr2=0x7f1eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb806c8f0 0x7f1eb806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ed806dd60 tx=0x7f1ec0009060 comp rx=0 tx=0).stop 2024-04-26T20:22:36.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 -- 172.21.15.37:0/4138215575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80d3820 msgr2=0x7f1ed80ba7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80d3820 0x7f1ed80ba7d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800c7f0 tx=0x7f1ec800cbf0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 -- 172.21.15.37:0/4138215575 shutdown_connections 2024-04-26T20:22:36.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1eb806c8f0 0x7f1eb806edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ed8162b40 0x7f1ed80cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ed80d3c80 0x7f1ed814d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 --2- 172.21.15.37:0/4138215575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80d3820 0x7f1ed80ba7d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 -- 172.21.15.37:0/4138215575 >> 172.21.15.37:0/4138215575 conn(0x7f1ed809e550 msgr2=0x7f1ed80d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:36.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 -- 172.21.15.37:0/4138215575 shutdown_connections 2024-04-26T20:22:36.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.107+0000 7f1edd7f7640 1 -- 172.21.15.37:0/4138215575 wait complete. 2024-04-26T20:22:36.126 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T20:22:36.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.227+0000 7fe872df7640 1 Processor -- start 2024-04-26T20:22:36.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.227+0000 7fe872df7640 1 -- start start 2024-04-26T20:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe872df7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe86c0cf810 0x7fe86c16a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe872df7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe86c0d2020 con 0x7fe86c06e530 2024-04-26T20:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe872df7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe86c0ce700 con 0x7fe86c0d1c40 2024-04-26T20:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe872df7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe86c0ce4b0 con 0x7fe86c0cf810 2024-04-26T20:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe86c06e530 0x7fe86c0cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33054/0 (socket says 172.21.15.37:33054) 2024-04-26T20:22:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86bfff640 1 -- 172.21.15.37:0/107424752 learned_addr learned my addr 172.21.15.37:0/107424752 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 313181097 0 0) 0x7fe86c0d2020 con 0x7fe86c06e530 2024-04-26T20:22:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe85c003520 con 0x7fe86c06e530 2024-04-26T20:22:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe871394640 1 --2- 172.21.15.37:0/107424752 >> v2:172.21.15.134:3300/0 conn(0x7fe86c0cf810 0x7fe86c16a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4062147175 0 0) 0x7fe85c003520 con 0x7fe86c06e530 2024-04-26T20:22:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 >> v2:172.21.15.134:3300/0 conn(0x7fe86c0cf810 msgr2=0x7fe86c16a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 --2- 172.21.15.37:0/107424752 >> v2:172.21.15.134:3300/0 conn(0x7fe86c0cf810 0x7fe86c16a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 >> v1:172.21.15.134:6789/0 conn(0x7fe86c0d1c40 legacy=0x7fe86c06fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c0ce9e0 con 0x7fe86c06e530 2024-04-26T20:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fe858001670 con 0x7fe86c06e530 2024-04-26T20:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 >> v1:172.21.15.37:6789/0 conn(0x7fe86c06e530 legacy=0x7fe86c0cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 --2- 172.21.15.37:0/107424752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe85c0039b0 0x7fe85c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 --2- 172.21.15.37:0/107424752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe85c004800 0x7fe85c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 --2- 172.21.15.37:0/107424752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe85c0051c0 0x7fe85c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c0ce4b0 con 0x7fe85c0039b0 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c00c110 con 0x7fe85c0051c0 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe85c00c310 con 0x7fe85c004800 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.228+0000 7fe870b93640 1 --2- 172.21.15.37:0/107424752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe85c0039b0 0x7fe85c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.229+0000 7fe86bfff640 1 --2- 172.21.15.37:0/107424752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe85c004800 0x7fe85c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.229+0000 7fe871394640 1 --2- 172.21.15.37:0/107424752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe85c0051c0 0x7fe85c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.229+0000 7fe870b93640 1 -- 172.21.15.37:0/107424752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe85c004800 msgr2=0x7fe85c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.229+0000 7fe870b93640 1 --2- 172.21.15.37:0/107424752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe85c004800 0x7fe85c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.229+0000 7fe870b93640 1 -- 172.21.15.37:0/107424752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe85c0051c0 msgr2=0x7fe85c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.229+0000 7fe870b93640 1 --2- 172.21.15.37:0/107424752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe85c0051c0 0x7fe85c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.229+0000 7fe870b93640 1 -- 172.21.15.37:0/107424752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85c003520 con 0x7fe85c0039b0 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.229+0000 7fe871394640 1 -- 172.21.15.37:0/107424752 reap_dead start 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.229+0000 7fe870b93640 1 --2- 172.21.15.37:0/107424752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe85c0039b0 0x7fe85c003dd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe860001750 tx=0x7fe85c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e32edd24a6bd6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.229+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860001370 con 0x7fe85c0039b0 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.230+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8600052d0 con 0x7fe85c0039b0 2024-04-26T20:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.230+0000 7fe86b7fe640 1 -- 172.21.15.37:0/107424752 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86002abc0 con 0x7fe85c0039b0 2024-04-26T20:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.230+0000 7fe872df7640 1 -- 172.21.15.37:0/107424752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe85c0039b0 msgr2=0x7fe85c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.230+0000 7fe872df7640 1 --2- 172.21.15.37:0/107424752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe85c0039b0 0x7fe85c003dd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe860001750 tx=0x7fe85c005830 comp rx=0 tx=0).stop 2024-04-26T20:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.230+0000 7fe872df7640 1 -- 172.21.15.37:0/107424752 shutdown_connections 2024-04-26T20:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.230+0000 7fe872df7640 1 --2- 172.21.15.37:0/107424752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe85c0039b0 0x7fe85c003dd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.230+0000 7fe872df7640 1 -- 172.21.15.37:0/107424752 >> 172.21.15.37:0/107424752 conn(0x7fe86c073880 msgr2=0x7fe86c073c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.230+0000 7fe872df7640 1 -- 172.21.15.37:0/107424752 shutdown_connections 2024-04-26T20:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.230+0000 7fe872df7640 1 -- 172.21.15.37:0/107424752 wait complete. 2024-04-26T20:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.231+0000 7fe872df7640 1 Processor -- start 2024-04-26T20:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.231+0000 7fe872df7640 1 -- start start 2024-04-26T20:22:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.232+0000 7fe872df7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe85c0039b0 0x7fe86c15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.232+0000 7fe872df7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe86c158580 0x7fe86c0c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.232+0000 7fe872df7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c154ee0 0x7fe86c0c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.232+0000 7fe872df7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c0cbb30 con 0x7fe86c154ee0 2024-04-26T20:22:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.232+0000 7fe872df7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c0cb850 con 0x7fe85c0039b0 2024-04-26T20:22:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.232+0000 7fe872df7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe86c0cf100 con 0x7fe86c158580 2024-04-26T20:22:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.232+0000 7fe871394640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c154ee0 0x7fe86c0c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.232+0000 7fe871394640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c154ee0 0x7fe86c0c7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48168/0 (socket says 172.21.15.37:48168) 2024-04-26T20:22:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.232+0000 7fe871394640 1 -- 172.21.15.37:0/1173027819 learned_addr learned my addr 172.21.15.37:0/1173027819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.232+0000 7fe86bfff640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe86c158580 0x7fe86c0c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.232+0000 7fe870b93640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe85c0039b0 0x7fe86c15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe871394640 1 -- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe86c158580 msgr2=0x7fe86c0c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe871394640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe86c158580 0x7fe86c0c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe871394640 1 -- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe85c0039b0 msgr2=0x7fe86c15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe871394640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe85c0039b0 0x7fe86c15ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe871394640 1 -- 172.21.15.37:0/1173027819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c0c7ef0 con 0x7fe86c154ee0 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe86bfff640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe86c158580 0x7fe86c0c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe870b93640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe85c0039b0 0x7fe86c15ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe871394640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c154ee0 0x7fe86c0c7400 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe864009840 tx=0x7fe864009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe8697fa640 1 -- 172.21.15.37:0/1173027819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864007cc0 con 0x7fe86c154ee0 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe8697fa640 1 -- 172.21.15.37:0/1173027819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe864019070 con 0x7fe86c154ee0 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe872df7640 1 -- 172.21.15.37:0/1173027819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86c0af010 con 0x7fe86c154ee0 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe8697fa640 1 -- 172.21.15.37:0/1173027819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864007e60 con 0x7fe86c154ee0 2024-04-26T20:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.233+0000 7fe872df7640 1 -- 172.21.15.37:0/1173027819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86c158060 con 0x7fe86c154ee0 2024-04-26T20:22:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.234+0000 7fe872df7640 1 -- 172.21.15.37:0/1173027819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe86c06a360 con 0x7fe86c154ee0 2024-04-26T20:22:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.235+0000 7fe8697fa640 1 -- 172.21.15.37:0/1173027819 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe864014650 con 0x7fe86c154ee0 2024-04-26T20:22:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.235+0000 7fe8697fa640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe85006c8f0 0x7fe85006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.235+0000 7fe8697fa640 1 -- 172.21.15.37:0/1173027819 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fe86408bea0 con 0x7fe86c154ee0 2024-04-26T20:22:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.235+0000 7fe870b93640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe85006c8f0 0x7fe85006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.236+0000 7fe870b93640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe85006c8f0 0x7fe85006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe860032000 tx=0x7fe86002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.239+0000 7fe8697fa640 1 -- 172.21.15.37:0/1173027819 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe864023020 con 0x7fe86c154ee0 2024-04-26T20:22:36.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.502+0000 7fe872df7640 1 -- 172.21.15.37:0/1173027819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fe86c0cc110 con 0x7fe86c154ee0 2024-04-26T20:22:36.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.504+0000 7fe8697fa640 1 -- 172.21.15.37:0/1173027819 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fe86405a620 con 0x7fe86c154ee0 2024-04-26T20:22:36.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.506+0000 7fe872df7640 1 -- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe85006c8f0 msgr2=0x7fe85006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.506+0000 7fe872df7640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe85006c8f0 0x7fe85006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe860032000 tx=0x7fe86002e000 comp rx=0 tx=0).stop 2024-04-26T20:22:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.506+0000 7fe872df7640 1 -- 172.21.15.37:0/1173027819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c154ee0 msgr2=0x7fe86c0c7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.506+0000 7fe872df7640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c154ee0 0x7fe86c0c7400 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe864009840 tx=0x7fe864009d00 comp rx=0 tx=0).stop 2024-04-26T20:22:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.506+0000 7fe872df7640 1 -- 172.21.15.37:0/1173027819 shutdown_connections 2024-04-26T20:22:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.506+0000 7fe872df7640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c154ee0 0x7fe86c0c7400 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.506+0000 7fe872df7640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe85006c8f0 0x7fe85006edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.506+0000 7fe872df7640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe86c158580 0x7fe86c0c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.506+0000 7fe872df7640 1 --2- 172.21.15.37:0/1173027819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe85c0039b0 0x7fe86c15ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.506+0000 7fe872df7640 1 -- 172.21.15.37:0/1173027819 >> 172.21.15.37:0/1173027819 conn(0x7fe86c073880 msgr2=0x7fe86c06fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.507+0000 7fe872df7640 1 -- 172.21.15.37:0/1173027819 shutdown_connections 2024-04-26T20:22:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.507+0000 7fe872df7640 1 -- 172.21.15.37:0/1173027819 wait complete. 2024-04-26T20:22:36.519 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T20:22:36.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.619+0000 7f664b609640 1 Processor -- start 2024-04-26T20:22:36.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.619+0000 7f664b609640 1 -- start start 2024-04-26T20:22:36.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f664b609640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6644161200 0x7f66440b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f664b609640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6644161b80 con 0x7f6644160d70 2024-04-26T20:22:36.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f664b609640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6644161690 con 0x7f664406f790 2024-04-26T20:22:36.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f664b609640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f66440b4400 con 0x7f6644161200 2024-04-26T20:22:36.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f6648ba4640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6644160d70 0x7f66440b5b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33056/0 (socket says 172.21.15.37:33056) 2024-04-26T20:22:36.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f6648ba4640 1 -- 172.21.15.37:0/2660467615 learned_addr learned my addr 172.21.15.37:0/2660467615 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:36.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180653694 0 0) 0x7f6644161b80 con 0x7f6644160d70 2024-04-26T20:22:36.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6630003520 con 0x7f6644160d70 2024-04-26T20:22:36.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f6649ba6640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6644161200 0x7f66440b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3761206769 0 0) 0x7f6630003520 con 0x7f6644160d70 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6644161200 msgr2=0x7f66440b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f663bfff640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6644161200 0x7f66440b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 >> v1:172.21.15.134:6789/0 conn(0x7f664406f790 legacy=0x7f6644074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66440b4120 con 0x7f6644160d70 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.620+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f6634001670 con 0x7f6644160d70 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 >> v1:172.21.15.37:6789/0 conn(0x7f6644160d70 legacy=0x7f66440b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f663bfff640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6644161200 0x7f66300039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f663bfff640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6630003f60 0x7f6630004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f663bfff640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6630004d40 0x7f66300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f66440b4400 con 0x7f6630003f60 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f663000baa0 con 0x7f6630004d40 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f663000bca0 con 0x7f6644161200 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f6648ba4640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6630003f60 0x7f6630004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f66493a5640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6644161200 0x7f66300039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f6649ba6640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6630004d40 0x7f66300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f6648ba4640 1 -- 172.21.15.37:0/2660467615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6644161200 msgr2=0x7f66300039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:36.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f6648ba4640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6644161200 0x7f66300039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f6648ba4640 1 -- 172.21.15.37:0/2660467615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6630004d40 msgr2=0x7f66300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:36.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f6648ba4640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6630004d40 0x7f66300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.621+0000 7f6648ba4640 1 -- 172.21.15.37:0/2660467615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6630003520 con 0x7f6630003f60 2024-04-26T20:22:36.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.622+0000 7f6648ba4640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6630003f60 0x7f6630004800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f663400c210 tx=0x7f6630005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24051ee1084bcc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:36.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.622+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634007d30 con 0x7f6630003f60 2024-04-26T20:22:36.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.622+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6634005af0 con 0x7f6630003f60 2024-04-26T20:22:36.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.622+0000 7f663bfff640 1 -- 172.21.15.37:0/2660467615 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634007070 con 0x7f6630003f60 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.622+0000 7f664b609640 1 -- 172.21.15.37:0/2660467615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6630003f60 msgr2=0x7f6630004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.622+0000 7f664b609640 1 --2- 172.21.15.37:0/2660467615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6630003f60 0x7f6630004800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f663400c210 tx=0x7f6630005830 comp rx=0 tx=0).stop 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.622+0000 7f6649ba6640 1 -- 172.21.15.37:0/2660467615 reap_dead start 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.622+0000 7f664b609640 1 -- 172.21.15.37:0/2660467615 shutdown_connections 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.622+0000 7f664b609640 1 -- 172.21.15.37:0/2660467615 >> 172.21.15.37:0/2660467615 conn(0x7f66440d0d50 msgr2=0x7f66440d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.623+0000 7f664b609640 1 -- 172.21.15.37:0/2660467615 shutdown_connections 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.623+0000 7f664b609640 1 -- 172.21.15.37:0/2660467615 wait complete. 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.624+0000 7f664b609640 1 Processor -- start 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.624+0000 7f664b609640 1 -- start start 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.624+0000 7f664b609640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6630003f60 0x7f664413c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.624+0000 7f664b609640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441451e0 0x7f6644145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.624+0000 7f664b609640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441413c0 0x7f6644141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.624+0000 7f664b609640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f66440735b0 con 0x7f66441413c0 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.624+0000 7f664b609640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66440732d0 con 0x7f66441451e0 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.624+0000 7f664b609640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f66440b3d80 con 0x7f6630003f60 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.624+0000 7f6649ba6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441413c0 0x7f6644141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.624+0000 7f6649ba6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441413c0 0x7f6644141840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48194/0 (socket says 172.21.15.37:48194) 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f6649ba6640 1 -- 172.21.15.37:0/235939898 learned_addr learned my addr 172.21.15.37:0/235939898 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f66493a5640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6630003f60 0x7f664413c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f6648ba4640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441451e0 0x7f6644145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f6649ba6640 1 -- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6630003f60 msgr2=0x7f664413c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f6649ba6640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6630003f60 0x7f664413c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f6649ba6640 1 -- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441451e0 msgr2=0x7f6644145660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f6649ba6640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441451e0 0x7f6644145660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f6649ba6640 1 -- 172.21.15.37:0/235939898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66441420f0 con 0x7f66441413c0 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f6649ba6640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441413c0 0x7f6644141840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6644076150 tx=0x7f663c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f6639ffb640 1 -- 172.21.15.37:0/235939898 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c00fdb0 con 0x7f66441413c0 2024-04-26T20:22:36.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f6639ffb640 1 -- 172.21.15.37:0/235939898 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f663c01d070 con 0x7f66441413c0 2024-04-26T20:22:36.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f664b609640 1 -- 172.21.15.37:0/235939898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66441423d0 con 0x7f66441413c0 2024-04-26T20:22:36.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f6639ffb640 1 -- 172.21.15.37:0/235939898 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c015670 con 0x7f66441413c0 2024-04-26T20:22:36.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.625+0000 7f664b609640 1 -- 172.21.15.37:0/235939898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6644142990 con 0x7f66441413c0 2024-04-26T20:22:36.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.626+0000 7f664b609640 1 -- 172.21.15.37:0/235939898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66440b49d0 con 0x7f66441413c0 2024-04-26T20:22:36.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.627+0000 7f6639ffb640 1 -- 172.21.15.37:0/235939898 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f663c015810 con 0x7f66441413c0 2024-04-26T20:22:36.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.627+0000 7f6639ffb640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f662806c920 0x7f662806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.627+0000 7f6639ffb640 1 -- 172.21.15.37:0/235939898 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f663c08c030 con 0x7f66441413c0 2024-04-26T20:22:36.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.627+0000 7f66493a5640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f662806c920 0x7f662806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.628+0000 7f66493a5640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f662806c920 0x7f662806ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f66400017e0 tx=0x7f6640009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:36.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.632+0000 7f6639ffb640 1 -- 172.21.15.37:0/235939898 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66440b49d0 con 0x7f66441413c0 2024-04-26T20:22:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.892+0000 7f664b609640 1 -- 172.21.15.37:0/235939898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f6644071b50 con 0x7f66441413c0 2024-04-26T20:22:36.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.896+0000 7f6639ffb640 1 -- 172.21.15.37:0/235939898 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f6644071b50 con 0x7f66441413c0 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 -- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f662806c920 msgr2=0x7f662806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f662806c920 0x7f662806ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f66400017e0 tx=0x7f6640009040 comp rx=0 tx=0).stop 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 -- 172.21.15.37:0/235939898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441413c0 msgr2=0x7f6644141840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441413c0 0x7f6644141840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6644076150 tx=0x7f663c00f580 comp rx=0 tx=0).stop 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 -- 172.21.15.37:0/235939898 shutdown_connections 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441413c0 0x7f6644141840 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441451e0 0x7f6644145660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f662806c920 0x7f662806ede0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 --2- 172.21.15.37:0/235939898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6630003f60 0x7f664413c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 -- 172.21.15.37:0/235939898 >> 172.21.15.37:0/235939898 conn(0x7f66440d0d50 msgr2=0x7f664415e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 -- 172.21.15.37:0/235939898 shutdown_connections 2024-04-26T20:22:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:36.898+0000 7f664b609640 1 -- 172.21.15.37:0/235939898 wait complete. 2024-04-26T20:22:36.910 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T20:22:37.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.011+0000 7faf72b0a640 1 Processor -- start 2024-04-26T20:22:37.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.011+0000 7faf72b0a640 1 -- start start 2024-04-26T20:22:37.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf72b0a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf6c159a90 0x7faf6c06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf72b0a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faf6c070130 0x7faf6c15fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf72b0a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faf6c160150 0x7faf6c15c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf72b0a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7faf6c160580 con 0x7faf6c070130 2024-04-26T20:22:37.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf72b0a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7faf6c072e30 con 0x7faf6c160150 2024-04-26T20:22:37.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf72b0a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faf6c073110 con 0x7faf6c159a90 2024-04-26T20:22:37.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf6bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faf6c070130 0x7faf6c15fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf6bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faf6c070130 0x7faf6c15fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48196/0 (socket says 172.21.15.37:48196) 2024-04-26T20:22:37.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf6bfff640 1 -- 172.21.15.37:0/3267647314 learned_addr learned my addr 172.21.15.37:0/3267647314 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:37.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf708a6640 1 --2- 172.21.15.37:0/3267647314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf6c159a90 0x7faf6c06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf710a7640 1 --2- 172.21.15.37:0/3267647314 >> v2:172.21.15.134:3300/0 conn(0x7faf6c160150 0x7faf6c15c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf6bfff640 1 -- 172.21.15.37:0/3267647314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf6c159a90 msgr2=0x7faf6c06fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf6bfff640 1 --2- 172.21.15.37:0/3267647314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf6c159a90 0x7faf6c06fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf6bfff640 1 -- 172.21.15.37:0/3267647314 >> v2:172.21.15.134:3300/0 conn(0x7faf6c160150 msgr2=0x7faf6c15c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf6bfff640 1 --2- 172.21.15.37:0/3267647314 >> v2:172.21.15.134:3300/0 conn(0x7faf6c160150 0x7faf6c15c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf6bfff640 1 -- 172.21.15.37:0/3267647314 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6c0733f0 con 0x7faf6c070130 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf708a6640 1 --2- 172.21.15.37:0/3267647314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf6c159a90 0x7faf6c06fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.012+0000 7faf710a7640 1 --2- 172.21.15.37:0/3267647314 >> v2:172.21.15.134:3300/0 conn(0x7faf6c160150 0x7faf6c15c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.013+0000 7faf6bfff640 1 --2- 172.21.15.37:0/3267647314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6c070130 0x7faf6c15fc10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faf6c073040 tx=0x7faf6002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19d8e82777bc332 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.013+0000 7faf6b7fe640 1 -- 172.21.15.37:0/3267647314 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60037070 con 0x7faf6c070130 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.013+0000 7faf6b7fe640 1 -- 172.21.15.37:0/3267647314 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6002adf0 con 0x7faf6c070130 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.013+0000 7faf6b7fe640 1 -- 172.21.15.37:0/3267647314 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60032660 con 0x7faf6c070130 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.014+0000 7faf72b0a640 1 -- 172.21.15.37:0/3267647314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6c070130 msgr2=0x7faf6c15fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.014+0000 7faf72b0a640 1 --2- 172.21.15.37:0/3267647314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6c070130 0x7faf6c15fc10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faf6c073040 tx=0x7faf6002a0a0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.014+0000 7faf72b0a640 1 -- 172.21.15.37:0/3267647314 shutdown_connections 2024-04-26T20:22:37.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.014+0000 7faf72b0a640 1 --2- 172.21.15.37:0/3267647314 >> v2:172.21.15.134:3300/0 conn(0x7faf6c160150 0x7faf6c15c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.014+0000 7faf72b0a640 1 --2- 172.21.15.37:0/3267647314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6c070130 0x7faf6c15fc10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.014+0000 7faf72b0a640 1 --2- 172.21.15.37:0/3267647314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf6c159a90 0x7faf6c06fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.014+0000 7faf72b0a640 1 -- 172.21.15.37:0/3267647314 >> 172.21.15.37:0/3267647314 conn(0x7faf6c0d0d50 msgr2=0x7faf6c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:37.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.014+0000 7faf72b0a640 1 -- 172.21.15.37:0/3267647314 shutdown_connections 2024-04-26T20:22:37.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.014+0000 7faf72b0a640 1 -- 172.21.15.37:0/3267647314 wait complete. 2024-04-26T20:22:37.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.015+0000 7faf72b0a640 1 Processor -- start 2024-04-26T20:22:37.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.015+0000 7faf72b0a640 1 -- start start 2024-04-26T20:22:37.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf72b0a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf6c070130 0x7faf6c14bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf72b0a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6c159a90 0x7faf6c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf72b0a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf6c160150 0x7faf6c0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf72b0a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c159e70 con 0x7faf6c159a90 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf72b0a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1609d0 con 0x7faf6c070130 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf72b0a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faf6c159ff0 con 0x7faf6c160150 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf6bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6c159a90 0x7faf6c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf6bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6c159a90 0x7faf6c0b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48204/0 (socket says 172.21.15.37:48204) 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf6bfff640 1 -- 172.21.15.37:0/1702523474 learned_addr learned my addr 172.21.15.37:0/1702523474 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf708a6640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf6c070130 0x7faf6c14bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf6bfff640 1 -- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf6c160150 msgr2=0x7faf6c0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf710a7640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf6c160150 0x7faf6c0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf6bfff640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf6c160150 0x7faf6c0ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf6bfff640 1 -- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf6c070130 msgr2=0x7faf6c14bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf6bfff640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf6c070130 0x7faf6c14bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf6bfff640 1 -- 172.21.15.37:0/1702523474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6c14c9e0 con 0x7faf6c159a90 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.016+0000 7faf708a6640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf6c070130 0x7faf6c14bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.017+0000 7faf6bfff640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6c159a90 0x7faf6c0b6300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faf60000c00 tx=0x7faf600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.017+0000 7faf697fa640 1 -- 172.21.15.37:0/1702523474 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60037040 con 0x7faf6c159a90 2024-04-26T20:22:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.017+0000 7faf697fa640 1 -- 172.21.15.37:0/1702523474 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf60042070 con 0x7faf6c159a90 2024-04-26T20:22:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.017+0000 7faf72b0a640 1 -- 172.21.15.37:0/1702523474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6c14c480 con 0x7faf6c159a90 2024-04-26T20:22:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.017+0000 7faf697fa640 1 -- 172.21.15.37:0/1702523474 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6003b850 con 0x7faf6c159a90 2024-04-26T20:22:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.017+0000 7faf72b0a640 1 -- 172.21.15.37:0/1702523474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf6c0b5260 con 0x7faf6c159a90 2024-04-26T20:22:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.018+0000 7faf72b0a640 1 -- 172.21.15.37:0/1702523474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6c06a360 con 0x7faf6c159a90 2024-04-26T20:22:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.018+0000 7faf697fa640 1 -- 172.21.15.37:0/1702523474 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf6003a070 con 0x7faf6c159a90 2024-04-26T20:22:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.019+0000 7faf697fa640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faf5806c920 0x7faf5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.019+0000 7faf697fa640 1 -- 172.21.15.37:0/1702523474 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7faf600ab8b0 con 0x7faf6c159a90 2024-04-26T20:22:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.019+0000 7faf708a6640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faf5806c920 0x7faf5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.020+0000 7faf708a6640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faf5806c920 0x7faf5806ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faf64008fc0 tx=0x7faf64008510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:37.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.023+0000 7faf697fa640 1 -- 172.21.15.37:0/1702523474 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf6c14c9e0 con 0x7faf6c159a90 2024-04-26T20:22:37.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.284+0000 7faf72b0a640 1 -- 172.21.15.37:0/1702523474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7faf6c158fa0 con 0x7faf6c159a90 2024-04-26T20:22:37.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.288+0000 7faf697fa640 1 -- 172.21.15.37:0/1702523474 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7faf60079fa0 con 0x7faf6c159a90 2024-04-26T20:22:37.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 -- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faf5806c920 msgr2=0x7faf5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faf5806c920 0x7faf5806ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faf64008fc0 tx=0x7faf64008510 comp rx=0 tx=0).stop 2024-04-26T20:22:37.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 -- 172.21.15.37:0/1702523474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6c159a90 msgr2=0x7faf6c0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6c159a90 0x7faf6c0b6300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faf60000c00 tx=0x7faf600046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 -- 172.21.15.37:0/1702523474 shutdown_connections 2024-04-26T20:22:37.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7faf5806c920 0x7faf5806ede0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf6c160150 0x7faf6c0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6c159a90 0x7faf6c0b6300 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 --2- 172.21.15.37:0/1702523474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf6c070130 0x7faf6c14bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 -- 172.21.15.37:0/1702523474 >> 172.21.15.37:0/1702523474 conn(0x7faf6c0d0d50 msgr2=0x7faf6c0cf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:37.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 -- 172.21.15.37:0/1702523474 shutdown_connections 2024-04-26T20:22:37.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.290+0000 7faf72b0a640 1 -- 172.21.15.37:0/1702523474 wait complete. 2024-04-26T20:22:37.302 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T20:22:37.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.403+0000 7fb65119e640 1 Processor -- start 2024-04-26T20:22:37.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.403+0000 7fb65119e640 1 -- start start 2024-04-26T20:22:37.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.404+0000 7fb65119e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb64c0d3820 0x7fb64c0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.404+0000 7fb65119e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb64c0d0210 con 0x7fb64c0b4ba0 2024-04-26T20:22:37.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.404+0000 7fb65119e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb64c0d04f0 con 0x7fb64c162b40 2024-04-26T20:22:37.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.404+0000 7fb65119e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb64c0d41c0 con 0x7fb64c0d3820 2024-04-26T20:22:37.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.404+0000 7fb64a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb64c0d3820 0x7fb64c0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.404+0000 7fb64b59e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb64c0b4ba0 0x7fb64c0b5030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33058/0 (socket says 172.21.15.37:33058) 2024-04-26T20:22:37.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.404+0000 7fb64b59e640 1 -- 172.21.15.37:0/4250625673 learned_addr learned my addr 172.21.15.37:0/4250625673 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:37.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.405+0000 7fb64a59c640 1 -- 172.21.15.37:0/4250625673 >> v1:172.21.15.134:6789/0 conn(0x7fb64c162b40 legacy=0x7fb64c0d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.405+0000 7fb64a59c640 1 -- 172.21.15.37:0/4250625673 >> v1:172.21.15.37:6789/0 conn(0x7fb64c0b4ba0 legacy=0x7fb64c0b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.405+0000 7fb64a59c640 1 -- 172.21.15.37:0/4250625673 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c0d07d0 con 0x7fb64c0d3820 2024-04-26T20:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.405+0000 7fb64a59c640 1 --2- 172.21.15.37:0/4250625673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb64c0d3820 0x7fb64c0d3c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb6340097a0 tx=0x7fb63402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3294349775c17179 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:37.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.405+0000 7fb649d9b640 1 -- 172.21.15.37:0/4250625673 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634037070 con 0x7fb64c0d3820 2024-04-26T20:22:37.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.406+0000 7fb649d9b640 1 -- 172.21.15.37:0/4250625673 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb634032440 con 0x7fb64c0d3820 2024-04-26T20:22:37.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.406+0000 7fb649d9b640 1 -- 172.21.15.37:0/4250625673 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634032810 con 0x7fb64c0d3820 2024-04-26T20:22:37.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.406+0000 7fb65119e640 1 -- 172.21.15.37:0/4250625673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb64c0d3820 msgr2=0x7fb64c0d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.406+0000 7fb65119e640 1 --2- 172.21.15.37:0/4250625673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb64c0d3820 0x7fb64c0d3c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb6340097a0 tx=0x7fb63402a230 comp rx=0 tx=0).stop 2024-04-26T20:22:37.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.406+0000 7fb65119e640 1 -- 172.21.15.37:0/4250625673 shutdown_connections 2024-04-26T20:22:37.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.406+0000 7fb65119e640 1 --2- 172.21.15.37:0/4250625673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb64c0d3820 0x7fb64c0d3c80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.406+0000 7fb65119e640 1 -- 172.21.15.37:0/4250625673 >> 172.21.15.37:0/4250625673 conn(0x7fb64c09e550 msgr2=0x7fb64c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:37.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.406+0000 7fb65119e640 1 -- 172.21.15.37:0/4250625673 shutdown_connections 2024-04-26T20:22:37.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.406+0000 7fb65119e640 1 -- 172.21.15.37:0/4250625673 wait complete. 2024-04-26T20:22:37.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.407+0000 7fb65119e640 1 Processor -- start 2024-04-26T20:22:37.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.408+0000 7fb65119e640 1 -- start start 2024-04-26T20:22:37.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.408+0000 7fb65119e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0b4ba0 0x7fb64c0cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.408+0000 7fb65119e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb64c0d3820 0x7fb64c0c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.408+0000 7fb65119e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb64c162b40 0x7fb64c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.408+0000 7fb65119e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c0d12b0 con 0x7fb64c0d3820 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.408+0000 7fb65119e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c0d0d90 con 0x7fb64c162b40 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.408+0000 7fb65119e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb64c0cfe30 con 0x7fb64c0b4ba0 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.408+0000 7fb64a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb64c0d3820 0x7fb64c0c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.408+0000 7fb64a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb64c0d3820 0x7fb64c0c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48236/0 (socket says 172.21.15.37:48236) 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.408+0000 7fb64a59c640 1 -- 172.21.15.37:0/1907840995 learned_addr learned my addr 172.21.15.37:0/1907840995 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb64b59e640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb64c162b40 0x7fb64c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb64ad9d640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0b4ba0 0x7fb64c0cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb64a59c640 1 -- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0b4ba0 msgr2=0x7fb64c0cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb64a59c640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0b4ba0 0x7fb64c0cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb64a59c640 1 -- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb64c162b40 msgr2=0x7fb64c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb64a59c640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb64c162b40 0x7fb64c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb64a59c640 1 -- 172.21.15.37:0/1907840995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c0cb8c0 con 0x7fb64c0d3820 2024-04-26T20:22:37.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb64a59c640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb64c0d3820 0x7fb64c0c8ba0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb634035040 tx=0x7fb6340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:37.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb6437fe640 1 -- 172.21.15.37:0/1907840995 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634037040 con 0x7fb64c0d3820 2024-04-26T20:22:37.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb6437fe640 1 -- 172.21.15.37:0/1907840995 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb634030770 con 0x7fb64c0d3820 2024-04-26T20:22:37.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb65119e640 1 -- 172.21.15.37:0/1907840995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64c0b83a0 con 0x7fb64c0d3820 2024-04-26T20:22:37.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb6437fe640 1 -- 172.21.15.37:0/1907840995 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634030b40 con 0x7fb64c0d3820 2024-04-26T20:22:37.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.409+0000 7fb65119e640 1 -- 172.21.15.37:0/1907840995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64c0b8960 con 0x7fb64c0d3820 2024-04-26T20:22:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.410+0000 7fb65119e640 1 -- 172.21.15.37:0/1907840995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64c06a360 con 0x7fb64c0d3820 2024-04-26T20:22:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.410+0000 7fb6437fe640 1 -- 172.21.15.37:0/1907840995 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb634044020 con 0x7fb64c0d3820 2024-04-26T20:22:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.411+0000 7fb6437fe640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb62c06c8f0 0x7fb62c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.411+0000 7fb6437fe640 1 -- 172.21.15.37:0/1907840995 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fb6340abdd0 con 0x7fb64c0d3820 2024-04-26T20:22:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.411+0000 7fb64ad9d640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb62c06c8f0 0x7fb62c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.412+0000 7fb64ad9d640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb62c06c8f0 0x7fb62c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb644001640 tx=0x7fb644009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:37.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.415+0000 7fb6437fe640 1 -- 172.21.15.37:0/1907840995 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb634032e20 con 0x7fb64c0d3820 2024-04-26T20:22:37.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.669+0000 7fb65119e640 1 -- 172.21.15.37:0/1907840995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fb64c0b1db0 con 0x7fb64c0d3820 2024-04-26T20:22:37.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.673+0000 7fb6437fe640 1 -- 172.21.15.37:0/1907840995 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fb63407a550 con 0x7fb64c0d3820 2024-04-26T20:22:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 -- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb62c06c8f0 msgr2=0x7fb62c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb62c06c8f0 0x7fb62c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb644001640 tx=0x7fb644009040 comp rx=0 tx=0).stop 2024-04-26T20:22:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 -- 172.21.15.37:0/1907840995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb64c0d3820 msgr2=0x7fb64c0c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb64c0d3820 0x7fb64c0c8ba0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb634035040 tx=0x7fb6340046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 -- 172.21.15.37:0/1907840995 shutdown_connections 2024-04-26T20:22:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb64c162b40 0x7fb64c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb64c0d3820 0x7fb64c0c8ba0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb62c06c8f0 0x7fb62c06edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 --2- 172.21.15.37:0/1907840995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0b4ba0 0x7fb64c0cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 -- 172.21.15.37:0/1907840995 >> 172.21.15.37:0/1907840995 conn(0x7fb64c09e550 msgr2=0x7fb64c0b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:37.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 -- 172.21.15.37:0/1907840995 shutdown_connections 2024-04-26T20:22:37.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.675+0000 7fb65119e640 1 -- 172.21.15.37:0/1907840995 wait complete. 2024-04-26T20:22:37.687 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T20:22:37.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.788+0000 7f83def6a640 1 Processor -- start 2024-04-26T20:22:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.788+0000 7f83def6a640 1 -- start start 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.789+0000 7f83def6a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f83d8162b40 0x7f83d80d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.789+0000 7f83def6a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d80d2670 con 0x7f83d80d3820 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.789+0000 7f83def6a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d80d0210 con 0x7f83d80d3c80 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.789+0000 7f83def6a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f83d80d04f0 con 0x7f83d8162b40 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.789+0000 7f83d7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f83d80d3820 0x7f83d80d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33074/0 (socket says 172.21.15.37:33074) 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.789+0000 7f83d7fff640 1 -- 172.21.15.37:0/2905609592 learned_addr learned my addr 172.21.15.37:0/2905609592 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.789+0000 7f83dcd06640 1 --2- 172.21.15.37:0/2905609592 >> v2:172.21.15.134:3300/0 conn(0x7f83d8162b40 0x7f83d80d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.790+0000 7f83d77fe640 1 -- 172.21.15.37:0/2905609592 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4156480079 0 0) 0x7f83d80d2670 con 0x7f83d80d3820 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.790+0000 7f83d77fe640 1 -- 172.21.15.37:0/2905609592 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83c0003520 con 0x7f83d80d3820 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.790+0000 7f83d77fe640 1 -- 172.21.15.37:0/2905609592 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15401290 0 0) 0x7f83d80d0210 con 0x7f83d80d3c80 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.790+0000 7f83d77fe640 1 -- 172.21.15.37:0/2905609592 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83d80d2670 con 0x7f83d80d3c80 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.790+0000 7f83dcd06640 1 -- 172.21.15.37:0/2905609592 >> v1:172.21.15.134:6789/0 conn(0x7f83d80d3c80 legacy=0x7f83d80d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.790+0000 7f83dcd06640 1 -- 172.21.15.37:0/2905609592 >> v1:172.21.15.37:6789/0 conn(0x7f83d80d3820 legacy=0x7f83d80d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.790+0000 7f83d77fe640 1 -- 172.21.15.37:0/2905609592 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2570871941 0 0) 0x7f83c0003520 con 0x7f83d80d3820 2024-04-26T20:22:37.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.790+0000 7f83dcd06640 1 -- 172.21.15.37:0/2905609592 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d80d07d0 con 0x7f83d8162b40 2024-04-26T20:22:37.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.791+0000 7f83dcd06640 1 --2- 172.21.15.37:0/2905609592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d8162b40 0x7f83d80d5bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009d90 tx=0x7f83d002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ce73193e933d28d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:37.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.791+0000 7f83d77fe640 1 -- 172.21.15.37:0/2905609592 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc001670 con 0x7f83d8162b40 2024-04-26T20:22:37.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.791+0000 7f83d77fe640 1 -- 172.21.15.37:0/2905609592 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d0007540 con 0x7f83d8162b40 2024-04-26T20:22:37.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.791+0000 7f83d77fe640 1 -- 172.21.15.37:0/2905609592 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d00078f0 con 0x7f83d8162b40 2024-04-26T20:22:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.791+0000 7f83def6a640 1 -- 172.21.15.37:0/2905609592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d8162b40 msgr2=0x7f83d80d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.791+0000 7f83def6a640 1 --2- 172.21.15.37:0/2905609592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d8162b40 0x7f83d80d5bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009d90 tx=0x7f83d002a710 comp rx=0 tx=0).stop 2024-04-26T20:22:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.791+0000 7f83def6a640 1 -- 172.21.15.37:0/2905609592 shutdown_connections 2024-04-26T20:22:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.791+0000 7f83def6a640 1 --2- 172.21.15.37:0/2905609592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d8162b40 0x7f83d80d5bd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.791+0000 7f83def6a640 1 -- 172.21.15.37:0/2905609592 >> 172.21.15.37:0/2905609592 conn(0x7f83d809e550 msgr2=0x7f83d80d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.791+0000 7f83def6a640 1 -- 172.21.15.37:0/2905609592 shutdown_connections 2024-04-26T20:22:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.791+0000 7f83def6a640 1 -- 172.21.15.37:0/2905609592 wait complete. 2024-04-26T20:22:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.793+0000 7f83def6a640 1 Processor -- start 2024-04-26T20:22:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.793+0000 7f83def6a640 1 -- start start 2024-04-26T20:22:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.793+0000 7f83def6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d80d3820 0x7f83d81526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.793+0000 7f83def6a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83d80d3c80 0x7f83d80cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.793+0000 7f83def6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d8162b40 0x7f83d80ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.793+0000 7f83def6a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f83d80d4110 con 0x7f83d80d3820 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.793+0000 7f83def6a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83d80d0d90 con 0x7f83d8162b40 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.793+0000 7f83def6a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f83d80b5bb0 con 0x7f83d80d3c80 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83dcd06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d80d3820 0x7f83d81526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83dcd06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d80d3820 0x7f83d81526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48242/0 (socket says 172.21.15.37:48242) 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83dcd06640 1 -- 172.21.15.37:0/3536694657 learned_addr learned my addr 172.21.15.37:0/3536694657 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83d7fff640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83d80d3c80 0x7f83d80cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83dd507640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d8162b40 0x7f83d80ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83dcd06640 1 -- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83d80d3c80 msgr2=0x7f83d80cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83dcd06640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83d80d3c80 0x7f83d80cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83dcd06640 1 -- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d8162b40 msgr2=0x7f83d80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83dcd06640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d8162b40 0x7f83d80ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83dcd06640 1 -- 172.21.15.37:0/3536694657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d8152bf0 con 0x7f83d80d3820 2024-04-26T20:22:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83d7fff640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83d80d3c80 0x7f83d80cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83dcd06640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d80d3820 0x7f83d81526b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009d90 tx=0x7f83d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83d57fa640 1 -- 172.21.15.37:0/3536694657 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0040070 con 0x7f83d80d3820 2024-04-26T20:22:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83d57fa640 1 -- 172.21.15.37:0/3536694657 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d0031c40 con 0x7f83d80d3820 2024-04-26T20:22:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83def6a640 1 -- 172.21.15.37:0/3536694657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d80c9540 con 0x7f83d80d3820 2024-04-26T20:22:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83d57fa640 1 -- 172.21.15.37:0/3536694657 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0009450 con 0x7f83d80d3820 2024-04-26T20:22:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.794+0000 7f83def6a640 1 -- 172.21.15.37:0/3536694657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d80c9b00 con 0x7f83d80d3820 2024-04-26T20:22:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.795+0000 7f83def6a640 1 -- 172.21.15.37:0/3536694657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d806a360 con 0x7f83d80d3820 2024-04-26T20:22:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.796+0000 7f83d57fa640 1 -- 172.21.15.37:0/3536694657 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83d0039040 con 0x7f83d80d3820 2024-04-26T20:22:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.796+0000 7f83d57fa640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83c406c8f0 0x7f83c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.797+0000 7f83d57fa640 1 -- 172.21.15.37:0/3536694657 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f83d00acd10 con 0x7f83d80d3820 2024-04-26T20:22:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.797+0000 7f83d7fff640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83c406c8f0 0x7f83c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.797+0000 7f83d7fff640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83c406c8f0 0x7f83c406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0017e0 tx=0x7f83cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:37.801+0000 7f83d57fa640 1 -- 172.21.15.37:0/3536694657 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f83d004b340 con 0x7f83d80d3820 2024-04-26T20:22:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.058+0000 7f83def6a640 1 -- 172.21.15.37:0/3536694657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f83d80bae20 con 0x7f83d80d3820 2024-04-26T20:22:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.062+0000 7f83d57fa640 1 -- 172.21.15.37:0/3536694657 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f83d007b490 con 0x7f83d80d3820 2024-04-26T20:22:38.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 -- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83c406c8f0 msgr2=0x7f83c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83c406c8f0 0x7f83c406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0017e0 tx=0x7f83cc009040 comp rx=0 tx=0).stop 2024-04-26T20:22:38.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 -- 172.21.15.37:0/3536694657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d80d3820 msgr2=0x7f83d81526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d80d3820 0x7f83d81526b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009d90 tx=0x7f83d0004a40 comp rx=0 tx=0).stop 2024-04-26T20:22:38.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 -- 172.21.15.37:0/3536694657 shutdown_connections 2024-04-26T20:22:38.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d8162b40 0x7f83d80ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83c406c8f0 0x7f83c406edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83d80d3c80 0x7f83d80cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 --2- 172.21.15.37:0/3536694657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d80d3820 0x7f83d81526b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 -- 172.21.15.37:0/3536694657 >> 172.21.15.37:0/3536694657 conn(0x7f83d809e550 msgr2=0x7f83d80b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 -- 172.21.15.37:0/3536694657 shutdown_connections 2024-04-26T20:22:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.064+0000 7f83def6a640 1 -- 172.21.15.37:0/3536694657 wait complete. 2024-04-26T20:22:38.076 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T20:22:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.176+0000 7fb5c7abd640 1 Processor -- start 2024-04-26T20:22:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.177+0000 7fb5c7abd640 1 -- start start 2024-04-26T20:22:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.177+0000 7fb5c7abd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c0162b40 0x7fb5c00d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.177+0000 7fb5c7abd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb5c00d3820 0x7fb5c00d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.177+0000 7fb5c7abd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb5c00d1850 0x7fb5c00d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.177+0000 7fb5c7abd640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c00d41c0 con 0x7fb5c00d3820 2024-04-26T20:22:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.177+0000 7fb5c7abd640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c00d0210 con 0x7fb5c00d1850 2024-04-26T20:22:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.177+0000 7fb5c7abd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c00d04f0 con 0x7fb5c0162b40 2024-04-26T20:22:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.177+0000 7fb5c5058640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb5c00d3820 0x7fb5c00d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.177+0000 7fb5c5058640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb5c00d3820 0x7fb5c00d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48244/0 (socket says 172.21.15.37:48244) 2024-04-26T20:22:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.177+0000 7fb5c5058640 1 -- 172.21.15.37:0/1320892937 learned_addr learned my addr 172.21.15.37:0/1320892937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.178+0000 7fb5c5058640 1 -- 172.21.15.37:0/1320892937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c0162b40 msgr2=0x7fb5c00d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.178+0000 7fb5c5859640 1 --2- 172.21.15.37:0/1320892937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c0162b40 0x7fb5c00d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.178+0000 7fb5c5058640 1 --2- 172.21.15.37:0/1320892937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c0162b40 0x7fb5c00d5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.178+0000 7fb5c605a640 1 --2- 172.21.15.37:0/1320892937 >> v2:172.21.15.134:3300/0 conn(0x7fb5c00d1850 0x7fb5c00d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.178+0000 7fb5c5058640 1 -- 172.21.15.37:0/1320892937 >> v2:172.21.15.134:3300/0 conn(0x7fb5c00d1850 msgr2=0x7fb5c00d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.178+0000 7fb5c5058640 1 --2- 172.21.15.37:0/1320892937 >> v2:172.21.15.134:3300/0 conn(0x7fb5c00d1850 0x7fb5c00d1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.178+0000 7fb5c5058640 1 -- 172.21.15.37:0/1320892937 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c00d07d0 con 0x7fb5c00d3820 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.178+0000 7fb5c5058640 1 --2- 172.21.15.37:0/1320892937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5c00d3820 0x7fb5c00d3c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00d43e0 tx=0x7fb5b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad7c40e8bb5783f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c4857640 1 -- 172.21.15.37:0/1320892937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4037070 con 0x7fb5c00d3820 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c4857640 1 -- 172.21.15.37:0/1320892937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b4032440 con 0x7fb5c00d3820 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c4857640 1 -- 172.21.15.37:0/1320892937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4032810 con 0x7fb5c00d3820 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1320892937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5c00d3820 msgr2=0x7fb5c00d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c7abd640 1 --2- 172.21.15.37:0/1320892937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5c00d3820 0x7fb5c00d3c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00d43e0 tx=0x7fb5b402a270 comp rx=0 tx=0).stop 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1320892937 shutdown_connections 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c7abd640 1 --2- 172.21.15.37:0/1320892937 >> v2:172.21.15.134:3300/0 conn(0x7fb5c00d1850 0x7fb5c00d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c7abd640 1 --2- 172.21.15.37:0/1320892937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5c00d3820 0x7fb5c00d3c80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c7abd640 1 --2- 172.21.15.37:0/1320892937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c0162b40 0x7fb5c00d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1320892937 >> 172.21.15.37:0/1320892937 conn(0x7fb5c009e550 msgr2=0x7fb5c00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1320892937 shutdown_connections 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.179+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1320892937 wait complete. 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.180+0000 7fb5c7abd640 1 Processor -- start 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.180+0000 7fb5c7abd640 1 -- start start 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.181+0000 7fb5c7abd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5c00d1850 0x7fb5c00ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.181+0000 7fb5c7abd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c00d3820 0x7fb5c01599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.181+0000 7fb5c7abd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5c0162b40 0x7fb5c00c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.181+0000 7fb5c7abd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00d6110 con 0x7fb5c00d1850 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.181+0000 7fb5c7abd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00d0d90 con 0x7fb5c0162b40 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.181+0000 7fb5c7abd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c00d12b0 con 0x7fb5c00d3820 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.181+0000 7fb5c5859640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5c00d1850 0x7fb5c00ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.181+0000 7fb5c5859640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5c00d1850 0x7fb5c00ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48260/0 (socket says 172.21.15.37:48260) 2024-04-26T20:22:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.181+0000 7fb5c5859640 1 -- 172.21.15.37:0/1483858603 learned_addr learned my addr 172.21.15.37:0/1483858603 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.181+0000 7fb5c5058640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c00d3820 0x7fb5c01599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.181+0000 7fb5c605a640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5c0162b40 0x7fb5c00c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.182+0000 7fb5c5859640 1 -- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c00d3820 msgr2=0x7fb5c01599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.182+0000 7fb5c5859640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c00d3820 0x7fb5c01599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.182+0000 7fb5c5859640 1 -- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5c0162b40 msgr2=0x7fb5c00c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.182+0000 7fb5c5859640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5c0162b40 0x7fb5c00c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.182+0000 7fb5c5859640 1 -- 172.21.15.37:0/1483858603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c00c9220 con 0x7fb5c00d1850 2024-04-26T20:22:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.182+0000 7fb5c5859640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5c00d1850 0x7fb5c00ca2c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c7f0 tx=0x7fb5b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.182+0000 7fb5be7fc640 1 -- 172.21.15.37:0/1483858603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8015070 con 0x7fb5c00d1850 2024-04-26T20:22:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.182+0000 7fb5be7fc640 1 -- 172.21.15.37:0/1483858603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b8004240 con 0x7fb5c00d1850 2024-04-26T20:22:38.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.182+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1483858603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c00c9500 con 0x7fb5c00d1850 2024-04-26T20:22:38.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.182+0000 7fb5be7fc640 1 -- 172.21.15.37:0/1483858603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8007620 con 0x7fb5c00d1850 2024-04-26T20:22:38.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.182+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1483858603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c00c9da0 con 0x7fb5c00d1850 2024-04-26T20:22:38.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.183+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1483858603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c006a360 con 0x7fb5c00d1850 2024-04-26T20:22:38.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.183+0000 7fb5be7fc640 1 -- 172.21.15.37:0/1483858603 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5b80079d0 con 0x7fb5c00d1850 2024-04-26T20:22:38.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.184+0000 7fb5be7fc640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5ac06c920 0x7fb5ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.184+0000 7fb5be7fc640 1 -- 172.21.15.37:0/1483858603 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fb5b808b7e0 con 0x7fb5c00d1850 2024-04-26T20:22:38.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.184+0000 7fb5c5058640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5ac06c920 0x7fb5ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.185+0000 7fb5c5058640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5ac06c920 0x7fb5ac06ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4034000 tx=0x7fb5b402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:38.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.188+0000 7fb5be7fc640 1 -- 172.21.15.37:0/1483858603 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5b8059ff0 con 0x7fb5c00d1850 2024-04-26T20:22:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.449+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1483858603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fb5c00d2960 con 0x7fb5c00d1850 2024-04-26T20:22:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.453+0000 7fb5be7fc640 1 -- 172.21.15.37:0/1483858603 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fb5b80598f0 con 0x7fb5c00d1850 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.455+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5ac06c920 msgr2=0x7fb5ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.455+0000 7fb5c7abd640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5ac06c920 0x7fb5ac06ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4034000 tx=0x7fb5b402e000 comp rx=0 tx=0).stop 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.455+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1483858603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5c00d1850 msgr2=0x7fb5c00ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.455+0000 7fb5c7abd640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5c00d1850 0x7fb5c00ca2c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c7f0 tx=0x7fb5b800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.455+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1483858603 shutdown_connections 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.455+0000 7fb5c7abd640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5c0162b40 0x7fb5c00c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.455+0000 7fb5c7abd640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5ac06c920 0x7fb5ac06ede0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.455+0000 7fb5c7abd640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c00d3820 0x7fb5c01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.455+0000 7fb5c7abd640 1 --2- 172.21.15.37:0/1483858603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5c00d1850 0x7fb5c00ca2c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.455+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1483858603 >> 172.21.15.37:0/1483858603 conn(0x7fb5c009e550 msgr2=0x7fb5c015ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.456+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1483858603 shutdown_connections 2024-04-26T20:22:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.456+0000 7fb5c7abd640 1 -- 172.21.15.37:0/1483858603 wait complete. 2024-04-26T20:22:38.467 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:38.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.607+0000 7f92d2af1640 1 Processor -- start 2024-04-26T20:22:38.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.608+0000 7f92d2af1640 1 -- start start 2024-04-26T20:22:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.608+0000 7f92d2af1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f92cc06ea40 0x7f92cc0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.608+0000 7f92d2af1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc0b1d70 con 0x7f92cc162e40 2024-04-26T20:22:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.608+0000 7f92d2af1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc0d1960 con 0x7f92cc0cf830 2024-04-26T20:22:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.608+0000 7f92d2af1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f92cc163220 con 0x7f92cc06ea40 2024-04-26T20:22:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.609+0000 7f92cbfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f92cc06ea40 0x7f92cc0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.609+0000 7f92d088d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f92cc162e40 0x7f92cc0b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33078/0 (socket says 172.21.15.37:33078) 2024-04-26T20:22:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.609+0000 7f92d088d640 1 -- 172.21.15.37:0/2482816114 learned_addr learned my addr 172.21.15.37:0/2482816114 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.609+0000 7f92cbfff640 1 -- 172.21.15.37:0/2482816114 >> v1:172.21.15.134:6789/0 conn(0x7f92cc0cf830 legacy=0x7f92cc0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.609+0000 7f92cbfff640 1 -- 172.21.15.37:0/2482816114 >> v1:172.21.15.37:6789/0 conn(0x7f92cc162e40 legacy=0x7f92cc0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.609+0000 7f92cbfff640 1 -- 172.21.15.37:0/2482816114 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc0d1c40 con 0x7f92cc06ea40 2024-04-26T20:22:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.610+0000 7f92cbfff640 1 --2- 172.21.15.37:0/2482816114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92cc06ea40 0x7f92cc0d1340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f92b80097a0 tx=0x7f92b802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638a292178fdb688 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.610+0000 7f92cb7fe640 1 -- 172.21.15.37:0/2482816114 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8037070 con 0x7f92cc06ea40 2024-04-26T20:22:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.610+0000 7f92cb7fe640 1 -- 172.21.15.37:0/2482816114 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b802ae50 con 0x7f92cc06ea40 2024-04-26T20:22:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.610+0000 7f92cb7fe640 1 -- 172.21.15.37:0/2482816114 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8032660 con 0x7f92cc06ea40 2024-04-26T20:22:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.610+0000 7f92d2af1640 1 -- 172.21.15.37:0/2482816114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92cc06ea40 msgr2=0x7f92cc0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.610+0000 7f92d2af1640 1 --2- 172.21.15.37:0/2482816114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92cc06ea40 0x7f92cc0d1340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f92b80097a0 tx=0x7f92b802a140 comp rx=0 tx=0).stop 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.610+0000 7f92d2af1640 1 -- 172.21.15.37:0/2482816114 shutdown_connections 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.610+0000 7f92d2af1640 1 --2- 172.21.15.37:0/2482816114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92cc06ea40 0x7f92cc0d1340 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.610+0000 7f92d2af1640 1 -- 172.21.15.37:0/2482816114 >> 172.21.15.37:0/2482816114 conn(0x7f92cc09ba50 msgr2=0x7f92cc09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.610+0000 7f92d2af1640 1 -- 172.21.15.37:0/2482816114 shutdown_connections 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.610+0000 7f92d2af1640 1 -- 172.21.15.37:0/2482816114 wait complete. 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.611+0000 7f92d2af1640 1 Processor -- start 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.612+0000 7f92d2af1640 1 -- start start 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.612+0000 7f92d2af1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92cc06ea40 0x7f92cc0ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.612+0000 7f92d2af1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92cc0cf830 0x7f92cc0b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.612+0000 7f92d2af1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc162e40 0x7f92cc1f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.612+0000 7f92d2af1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc0ce010 con 0x7f92cc0cf830 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.612+0000 7f92d2af1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc0d0eb0 con 0x7f92cc162e40 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.612+0000 7f92d2af1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f92cc0ce190 con 0x7f92cc06ea40 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.612+0000 7f92cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92cc0cf830 0x7f92cc0b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.612+0000 7f92cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92cc0cf830 0x7f92cc0b84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48272/0 (socket says 172.21.15.37:48272) 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.612+0000 7f92cbfff640 1 -- 172.21.15.37:0/2606910139 learned_addr learned my addr 172.21.15.37:0/2606910139 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92d108e640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc162e40 0x7f92cc1f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92d088d640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92cc06ea40 0x7f92cc0ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92cbfff640 1 -- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92cc06ea40 msgr2=0x7f92cc0ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92cbfff640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92cc06ea40 0x7f92cc0ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92cbfff640 1 -- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc162e40 msgr2=0x7f92cc1f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92cbfff640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc162e40 0x7f92cc1f8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92cbfff640 1 -- 172.21.15.37:0/2606910139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc0b40e0 con 0x7f92cc0cf830 2024-04-26T20:22:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92d108e640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc162e40 0x7f92cc1f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92cbfff640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92cc0cf830 0x7f92cc0b84f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f92b8000c00 tx=0x7f92b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92c97fa640 1 -- 172.21.15.37:0/2606910139 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8041070 con 0x7f92cc0cf830 2024-04-26T20:22:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92c97fa640 1 -- 172.21.15.37:0/2606910139 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b803c670 con 0x7f92cc0cf830 2024-04-26T20:22:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92d2af1640 1 -- 172.21.15.37:0/2606910139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92cc0b3320 con 0x7f92cc0cf830 2024-04-26T20:22:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92c97fa640 1 -- 172.21.15.37:0/2606910139 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8037040 con 0x7f92cc0cf830 2024-04-26T20:22:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.613+0000 7f92d2af1640 1 -- 172.21.15.37:0/2606910139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92cc0b38e0 con 0x7f92cc0cf830 2024-04-26T20:22:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.614+0000 7f92d2af1640 1 -- 172.21.15.37:0/2606910139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92cc06a360 con 0x7f92cc0cf830 2024-04-26T20:22:38.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.615+0000 7f92c97fa640 1 -- 172.21.15.37:0/2606910139 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92b8044070 con 0x7f92cc0cf830 2024-04-26T20:22:38.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.615+0000 7f92c97fa640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b006c920 0x7f92b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.615+0000 7f92c97fa640 1 -- 172.21.15.37:0/2606910139 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f92b80abf40 con 0x7f92cc0cf830 2024-04-26T20:22:38.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.615+0000 7f92d088d640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b006c920 0x7f92b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.616+0000 7f92d088d640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b006c920 0x7f92b006ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f92c0000f60 tx=0x7f92c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.619+0000 7f92c97fa640 1 -- 172.21.15.37:0/2606910139 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92b807a870 con 0x7f92cc0cf830 2024-04-26T20:22:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.810+0000 7f92d2af1640 1 -- 172.21.15.37:0/2606910139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92cc0b5920 con 0x7f92cc0cf830 2024-04-26T20:22:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.811+0000 7f92c97fa640 1 -- 172.21.15.37:0/2606910139 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12903 (secure 0 0 0) 0x7f92b80a8280 con 0x7f92cc0cf830 2024-04-26T20:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:22:35.718533+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.813+0000 7f92d2af1640 1 -- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b006c920 msgr2=0x7f92b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.813+0000 7f92d2af1640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b006c920 0x7f92b006ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f92c0000f60 tx=0x7f92c0009040 comp rx=0 tx=0).stop 2024-04-26T20:22:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.813+0000 7f92d2af1640 1 -- 172.21.15.37:0/2606910139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92cc0cf830 msgr2=0x7f92cc0b84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.813+0000 7f92d2af1640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92cc0cf830 0x7f92cc0b84f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f92b8000c00 tx=0x7f92b80046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.814+0000 7f92d2af1640 1 -- 172.21.15.37:0/2606910139 shutdown_connections 2024-04-26T20:22:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.814+0000 7f92d2af1640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc162e40 0x7f92cc1f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.814+0000 7f92d2af1640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92cc0cf830 0x7f92cc0b84f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.814+0000 7f92d2af1640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b006c920 0x7f92b006ede0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.814+0000 7f92d2af1640 1 --2- 172.21.15.37:0/2606910139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92cc06ea40 0x7f92cc0ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.814+0000 7f92d2af1640 1 -- 172.21.15.37:0/2606910139 >> 172.21.15.37:0/2606910139 conn(0x7f92cc09ba50 msgr2=0x7f92cc0d2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.814+0000 7f92d2af1640 1 -- 172.21.15.37:0/2606910139 shutdown_connections 2024-04-26T20:22:38.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.814+0000 7f92d2af1640 1 -- 172.21.15.37:0/2606910139 wait complete. 2024-04-26T20:22:38.826 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:22:38.826 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:22:38.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.929+0000 7efe6db6a640 1 Processor -- start 2024-04-26T20:22:38.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.929+0000 7efe6db6a640 1 -- start start 2024-04-26T20:22:38.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe6db6a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efe6815f400 0x7efe680b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe6db6a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe680d5a80 0x7efe680b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe6db6a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efe680b4b20 0x7efe680d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe6db6a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7efe680b43e0 con 0x7efe6815f400 2024-04-26T20:22:38.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe6db6a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7efe680b2760 con 0x7efe680b4b20 2024-04-26T20:22:38.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe6db6a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efe680b2a00 con 0x7efe680d5a80 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe677fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efe6815f400 0x7efe680b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe677fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efe6815f400 0x7efe680b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48274/0 (socket says 172.21.15.37:48274) 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe677fe640 1 -- 172.21.15.37:0/4079634551 learned_addr learned my addr 172.21.15.37:0/4079634551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe677fe640 1 -- 172.21.15.37:0/4079634551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe680d5a80 msgr2=0x7efe680b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe677fe640 1 --2- 172.21.15.37:0/4079634551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe680d5a80 0x7efe680b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe677fe640 1 -- 172.21.15.37:0/4079634551 >> v2:172.21.15.134:3300/0 conn(0x7efe680b4b20 msgr2=0x7efe680d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe67fff640 1 --2- 172.21.15.37:0/4079634551 >> v2:172.21.15.134:3300/0 conn(0x7efe680b4b20 0x7efe680d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe677fe640 1 --2- 172.21.15.37:0/4079634551 >> v2:172.21.15.134:3300/0 conn(0x7efe680b4b20 0x7efe680d3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.930+0000 7efe677fe640 1 -- 172.21.15.37:0/4079634551 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe680b2ce0 con 0x7efe6815f400 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.931+0000 7efe677fe640 1 --2- 172.21.15.37:0/4079634551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe6815f400 0x7efe680b37c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efe68074870 tx=0x7efe5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687d90f0f578ec17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.931+0000 7efe667fc640 1 -- 172.21.15.37:0/4079634551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58037070 con 0x7efe6815f400 2024-04-26T20:22:38.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.931+0000 7efe667fc640 1 -- 172.21.15.37:0/4079634551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe58032440 con 0x7efe6815f400 2024-04-26T20:22:38.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.931+0000 7efe667fc640 1 -- 172.21.15.37:0/4079634551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58032810 con 0x7efe6815f400 2024-04-26T20:22:38.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.932+0000 7efe6db6a640 1 -- 172.21.15.37:0/4079634551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe6815f400 msgr2=0x7efe680b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.932+0000 7efe6db6a640 1 --2- 172.21.15.37:0/4079634551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe6815f400 0x7efe680b37c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efe68074870 tx=0x7efe5802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.932+0000 7efe6db6a640 1 -- 172.21.15.37:0/4079634551 shutdown_connections 2024-04-26T20:22:38.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.932+0000 7efe6db6a640 1 --2- 172.21.15.37:0/4079634551 >> v2:172.21.15.134:3300/0 conn(0x7efe680b4b20 0x7efe680d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.932+0000 7efe6db6a640 1 --2- 172.21.15.37:0/4079634551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe680d5a80 0x7efe680b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.932+0000 7efe6db6a640 1 --2- 172.21.15.37:0/4079634551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe6815f400 0x7efe680b37c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.932+0000 7efe6db6a640 1 -- 172.21.15.37:0/4079634551 >> 172.21.15.37:0/4079634551 conn(0x7efe6809e4f0 msgr2=0x7efe680d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:38.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.932+0000 7efe6db6a640 1 -- 172.21.15.37:0/4079634551 shutdown_connections 2024-04-26T20:22:38.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.932+0000 7efe6db6a640 1 -- 172.21.15.37:0/4079634551 wait complete. 2024-04-26T20:22:38.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.933+0000 7efe6db6a640 1 Processor -- start 2024-04-26T20:22:38.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.933+0000 7efe6db6a640 1 -- start start 2024-04-26T20:22:38.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe6db6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe680b4b20 0x7efe6814d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe6db6a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe680d5a80 0x7efe6814a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe6db6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe6815f400 0x7efe680cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe6db6a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efe680b4f70 con 0x7efe680b4b20 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe6db6a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efe680b32a0 con 0x7efe6815f400 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe6db6a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efe680b3d00 con 0x7efe680d5a80 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe677fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe680b4b20 0x7efe6814d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe677fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe680b4b20 0x7efe6814d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48290/0 (socket says 172.21.15.37:48290) 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe677fe640 1 -- 172.21.15.37:0/2794954562 learned_addr learned my addr 172.21.15.37:0/2794954562 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe677fe640 1 -- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe680d5a80 msgr2=0x7efe6814a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe677fe640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe680d5a80 0x7efe6814a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe677fe640 1 -- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe6815f400 msgr2=0x7efe680cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe677fe640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe6815f400 0x7efe680cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe677fe640 1 -- 172.21.15.37:0/2794954562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6814ab40 con 0x7efe680b4b20 2024-04-26T20:22:38.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe677fe640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe680b4b20 0x7efe6814d210 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efe5802a220 tx=0x7efe580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:38.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.934+0000 7efe4ffff640 1 -- 172.21.15.37:0/2794954562 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58037040 con 0x7efe680b4b20 2024-04-26T20:22:38.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.935+0000 7efe4ffff640 1 -- 172.21.15.37:0/2794954562 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe58042070 con 0x7efe680b4b20 2024-04-26T20:22:38.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.935+0000 7efe6db6a640 1 -- 172.21.15.37:0/2794954562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe680b93d0 con 0x7efe680b4b20 2024-04-26T20:22:38.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.935+0000 7efe4ffff640 1 -- 172.21.15.37:0/2794954562 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58030bb0 con 0x7efe680b4b20 2024-04-26T20:22:38.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.935+0000 7efe6db6a640 1 -- 172.21.15.37:0/2794954562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe680b9930 con 0x7efe680b4b20 2024-04-26T20:22:38.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.936+0000 7efe6db6a640 1 -- 172.21.15.37:0/2794954562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6806a360 con 0x7efe680b4b20 2024-04-26T20:22:38.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.936+0000 7efe4ffff640 1 -- 172.21.15.37:0/2794954562 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe58047460 con 0x7efe680b4b20 2024-04-26T20:22:38.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.936+0000 7efe4ffff640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe5406c8f0 0x7efe5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.937+0000 7efe4ffff640 1 -- 172.21.15.37:0/2794954562 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7efe580ac470 con 0x7efe680b4b20 2024-04-26T20:22:38.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.937+0000 7efe66ffd640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe5406c8f0 0x7efe5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.937+0000 7efe66ffd640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe5406c8f0 0x7efe5406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009670 tx=0x7efe5c006bf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:38.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:38.941+0000 7efe4ffff640 1 -- 172.21.15.37:0/2794954562 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe680b93d0 con 0x7efe680b4b20 2024-04-26T20:22:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.123+0000 7efe6db6a640 1 -- 172.21.15.37:0/2794954562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe680d28f0 con 0x7efe680b4b20 2024-04-26T20:22:39.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.124+0000 7efe4ffff640 1 -- 172.21.15.37:0/2794954562 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7efe58045360 con 0x7efe680b4b20 2024-04-26T20:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:22:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.125+0000 7efe6db6a640 1 -- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe5406c8f0 msgr2=0x7efe5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.125+0000 7efe6db6a640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe5406c8f0 0x7efe5406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009670 tx=0x7efe5c006bf0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.125+0000 7efe6db6a640 1 -- 172.21.15.37:0/2794954562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe680b4b20 msgr2=0x7efe6814d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.125+0000 7efe6db6a640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe680b4b20 0x7efe6814d210 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efe5802a220 tx=0x7efe580046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.125+0000 7efe6db6a640 1 -- 172.21.15.37:0/2794954562 shutdown_connections 2024-04-26T20:22:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.125+0000 7efe6db6a640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe6815f400 0x7efe680cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.125+0000 7efe6db6a640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe5406c8f0 0x7efe5406edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.125+0000 7efe6db6a640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe680d5a80 0x7efe6814a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.125+0000 7efe6db6a640 1 --2- 172.21.15.37:0/2794954562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe680b4b20 0x7efe6814d210 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.125+0000 7efe6db6a640 1 -- 172.21.15.37:0/2794954562 >> 172.21.15.37:0/2794954562 conn(0x7efe6809e4f0 msgr2=0x7efe680d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:39.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.126+0000 7efe6db6a640 1 -- 172.21.15.37:0/2794954562 shutdown_connections 2024-04-26T20:22:39.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.126+0000 7efe6db6a640 1 -- 172.21.15.37:0/2794954562 wait complete. 2024-04-26T20:22:39.138 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:22:39.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.238+0000 7f5fea55f640 1 Processor -- start 2024-04-26T20:22:39.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.238+0000 7f5fea55f640 1 -- start start 2024-04-26T20:22:39.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fea55f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5fe406c240 0x7f5fe406fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fea55f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5fe415f4f0 0x7f5fe406c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fea55f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe406cb60 con 0x7f5fe40b1480 2024-04-26T20:22:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fea55f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe40b59b0 con 0x7f5fe406c240 2024-04-26T20:22:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fea55f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe4070350 con 0x7f5fe415f4f0 2024-04-26T20:22:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fe3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5fe406c240 0x7f5fe406fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fe8afc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5fe40b1480 0x7f5fe40ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33086/0 (socket says 172.21.15.37:33086) 2024-04-26T20:22:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fe8afc640 1 -- 172.21.15.37:0/1398883357 learned_addr learned my addr 172.21.15.37:0/1398883357 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fe37fe640 1 --2- 172.21.15.37:0/1398883357 >> v2:172.21.15.134:3300/0 conn(0x7f5fe415f4f0 0x7f5fe406c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fe3fff640 1 -- 172.21.15.37:0/1398883357 >> v2:172.21.15.134:3300/0 conn(0x7f5fe415f4f0 msgr2=0x7f5fe406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fe3fff640 1 --2- 172.21.15.37:0/1398883357 >> v2:172.21.15.134:3300/0 conn(0x7f5fe415f4f0 0x7f5fe406c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fe3fff640 1 -- 172.21.15.37:0/1398883357 >> v1:172.21.15.37:6789/0 conn(0x7f5fe40b1480 legacy=0x7f5fe40ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.239+0000 7f5fe3fff640 1 -- 172.21.15.37:0/1398883357 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe40b5c70 con 0x7f5fe406c240 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.240+0000 7f5fe3fff640 1 --2- 172.21.15.37:0/1398883357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fe406c240 0x7f5fe406fd70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40b5900 tx=0x7f5fcc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6295ffbb4172f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.240+0000 7f5fe2ffd640 1 -- 172.21.15.37:0/1398883357 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc037070 con 0x7f5fe406c240 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.240+0000 7f5fe2ffd640 1 -- 172.21.15.37:0/1398883357 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fcc032440 con 0x7f5fe406c240 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.240+0000 7f5fe2ffd640 1 -- 172.21.15.37:0/1398883357 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc032810 con 0x7f5fe406c240 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.241+0000 7f5fea55f640 1 -- 172.21.15.37:0/1398883357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fe406c240 msgr2=0x7f5fe406fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.241+0000 7f5fea55f640 1 --2- 172.21.15.37:0/1398883357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fe406c240 0x7f5fe406fd70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40b5900 tx=0x7f5fcc02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.241+0000 7f5fea55f640 1 -- 172.21.15.37:0/1398883357 shutdown_connections 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.241+0000 7f5fea55f640 1 --2- 172.21.15.37:0/1398883357 >> v2:172.21.15.134:3300/0 conn(0x7f5fe415f4f0 0x7f5fe406c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.241+0000 7f5fea55f640 1 --2- 172.21.15.37:0/1398883357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fe406c240 0x7f5fe406fd70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.241+0000 7f5fea55f640 1 -- 172.21.15.37:0/1398883357 >> 172.21.15.37:0/1398883357 conn(0x7f5fe40b3b70 msgr2=0x7f5fe40b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.241+0000 7f5fea55f640 1 -- 172.21.15.37:0/1398883357 shutdown_connections 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.241+0000 7f5fea55f640 1 -- 172.21.15.37:0/1398883357 wait complete. 2024-04-26T20:22:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.242+0000 7f5fea55f640 1 Processor -- start 2024-04-26T20:22:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.242+0000 7f5fea55f640 1 -- start start 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fea55f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fe406c240 0x7f5fe40d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fea55f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fe40b1480 0x7f5fe41f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fea55f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fe415f4f0 0x7f5fe40b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fea55f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe40b0fc0 con 0x7f5fe415f4f0 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fea55f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe40b6230 con 0x7f5fe40b1480 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fea55f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe40b6750 con 0x7f5fe406c240 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe8afc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fe415f4f0 0x7f5fe40b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe8afc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fe415f4f0 0x7f5fe40b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48316/0 (socket says 172.21.15.37:48316) 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe8afc640 1 -- 172.21.15.37:0/2985039712 learned_addr learned my addr 172.21.15.37:0/2985039712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe3fff640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fe406c240 0x7f5fe40d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe37fe640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fe40b1480 0x7f5fe41f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe8afc640 1 -- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fe406c240 msgr2=0x7f5fe40d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe8afc640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fe406c240 0x7f5fe40d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe8afc640 1 -- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fe40b1480 msgr2=0x7f5fe41f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe8afc640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fe40b1480 0x7f5fe41f7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe8afc640 1 -- 172.21.15.37:0/2985039712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe40d0fb0 con 0x7f5fe415f4f0 2024-04-26T20:22:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe3fff640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fe406c240 0x7f5fe40d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe8afc640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fe415f4f0 0x7f5fe40b2950 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00cbc0 tx=0x7f5fdc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.243+0000 7f5fe0ff9640 1 -- 172.21.15.37:0/2985039712 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc017070 con 0x7f5fe415f4f0 2024-04-26T20:22:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.244+0000 7f5fe0ff9640 1 -- 172.21.15.37:0/2985039712 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fdc00ad40 con 0x7f5fe415f4f0 2024-04-26T20:22:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.244+0000 7f5fea55f640 1 -- 172.21.15.37:0/2985039712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe4156630 con 0x7f5fe415f4f0 2024-04-26T20:22:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.244+0000 7f5fe0ff9640 1 -- 172.21.15.37:0/2985039712 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc012600 con 0x7f5fe415f4f0 2024-04-26T20:22:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.244+0000 7f5fea55f640 1 -- 172.21.15.37:0/2985039712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe4156ed0 con 0x7f5fe415f4f0 2024-04-26T20:22:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.244+0000 7f5fea55f640 1 -- 172.21.15.37:0/2985039712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe406a360 con 0x7f5fe415f4f0 2024-04-26T20:22:39.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.245+0000 7f5fe0ff9640 1 -- 172.21.15.37:0/2985039712 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fdc01a070 con 0x7f5fe415f4f0 2024-04-26T20:22:39.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.245+0000 7f5fe0ff9640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5fc406c8f0 0x7f5fc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.246+0000 7f5fe0ff9640 1 -- 172.21.15.37:0/2985039712 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f5fdc08b440 con 0x7f5fe415f4f0 2024-04-26T20:22:39.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.246+0000 7f5fe3fff640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5fc406c8f0 0x7f5fc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.246+0000 7f5fe3fff640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5fc406c8f0 0x7f5fc406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc034000 tx=0x7f5fcc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.250+0000 7f5fe0ff9640 1 -- 172.21.15.37:0/2985039712 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5fdc059600 con 0x7f5fe415f4f0 2024-04-26T20:22:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.439+0000 7f5fea55f640 1 -- 172.21.15.37:0/2985039712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5fe406fee0 con 0x7f5fe415f4f0 2024-04-26T20:22:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.439+0000 7f5fe0ff9640 1 -- 172.21.15.37:0/2985039712 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f5fdc05d900 con 0x7f5fe415f4f0 2024-04-26T20:22:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-26T20:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-26T20:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-26T20:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-26T20:22:39.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 -- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5fc406c8f0 msgr2=0x7f5fc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5fc406c8f0 0x7f5fc406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc034000 tx=0x7f5fcc02e000 comp rx=0 tx=0).stop 2024-04-26T20:22:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 -- 172.21.15.37:0/2985039712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fe415f4f0 msgr2=0x7f5fe40b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fe415f4f0 0x7f5fe40b2950 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00cbc0 tx=0x7f5fdc00a430 comp rx=0 tx=0).stop 2024-04-26T20:22:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 -- 172.21.15.37:0/2985039712 shutdown_connections 2024-04-26T20:22:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fe415f4f0 0x7f5fe40b2950 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fe40b1480 0x7f5fe41f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5fc406c8f0 0x7f5fc406edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 --2- 172.21.15.37:0/2985039712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fe406c240 0x7f5fe40d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 -- 172.21.15.37:0/2985039712 >> 172.21.15.37:0/2985039712 conn(0x7f5fe40b3b70 msgr2=0x7f5fe415e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 -- 172.21.15.37:0/2985039712 shutdown_connections 2024-04-26T20:22:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.441+0000 7f5fea55f640 1 -- 172.21.15.37:0/2985039712 wait complete. 2024-04-26T20:22:39.453 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:22:39.453 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:22:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.558+0000 7f561d904640 1 Processor -- start 2024-04-26T20:22:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.558+0000 7f561d904640 1 -- start start 2024-04-26T20:22:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f561d904640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f561815f410 0x7f56180b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f561d904640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f56180d3e80 0x7f56180d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f561d904640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56180b55a0 con 0x7f561815f7f0 2024-04-26T20:22:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f561d904640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f56180b5880 con 0x7f56180d3e80 2024-04-26T20:22:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f561d904640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f56180b5b60 con 0x7f561815f410 2024-04-26T20:22:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f56177fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f56180d3e80 0x7f56180d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f56167fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f561815f7f0 0x7f56180d3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33098/0 (socket says 172.21.15.37:33098) 2024-04-26T20:22:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f56167fc640 1 -- 172.21.15.37:0/984236730 learned_addr learned my addr 172.21.15.37:0/984236730 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f5616ffd640 1 --2- 172.21.15.37:0/984236730 >> v2:172.21.15.134:3300/0 conn(0x7f561815f410 0x7f56180b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f56177fe640 1 -- 172.21.15.37:0/984236730 >> v2:172.21.15.134:3300/0 conn(0x7f561815f410 msgr2=0x7f56180b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f56177fe640 1 --2- 172.21.15.37:0/984236730 >> v2:172.21.15.134:3300/0 conn(0x7f561815f410 0x7f56180b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f56177fe640 1 -- 172.21.15.37:0/984236730 >> v1:172.21.15.37:6789/0 conn(0x7f561815f7f0 legacy=0x7f56180d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f56177fe640 1 -- 172.21.15.37:0/984236730 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56180b5e40 con 0x7f56180d3e80 2024-04-26T20:22:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f5616ffd640 1 --2- 172.21.15.37:0/984236730 >> v2:172.21.15.134:3300/0 conn(0x7f561815f410 0x7f56180b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.559+0000 7f5615ffb640 1 -- 172.21.15.37:0/984236730 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4226867669 0 0) 0x7f56180b55a0 con 0x7f561815f7f0 2024-04-26T20:22:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.560+0000 7f56177fe640 1 --2- 172.21.15.37:0/984236730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56180d3e80 0x7f56180d0110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f561806cc00 tx=0x7f561002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519dc56e53a3f7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.560+0000 7f5615ffb640 1 -- 172.21.15.37:0/984236730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610007740 con 0x7f56180d3e80 2024-04-26T20:22:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.560+0000 7f5615ffb640 1 -- 172.21.15.37:0/984236730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56100078e0 con 0x7f56180d3e80 2024-04-26T20:22:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.560+0000 7f5615ffb640 1 -- 172.21.15.37:0/984236730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610007cb0 con 0x7f56180d3e80 2024-04-26T20:22:39.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.561+0000 7f561d904640 1 -- 172.21.15.37:0/984236730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56180d3e80 msgr2=0x7f56180d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.561+0000 7f561d904640 1 --2- 172.21.15.37:0/984236730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56180d3e80 0x7f56180d0110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f561806cc00 tx=0x7f561002a710 comp rx=0 tx=0).stop 2024-04-26T20:22:39.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.561+0000 7f561d904640 1 -- 172.21.15.37:0/984236730 shutdown_connections 2024-04-26T20:22:39.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.561+0000 7f561d904640 1 --2- 172.21.15.37:0/984236730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56180d3e80 0x7f56180d0110 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.561+0000 7f561d904640 1 --2- 172.21.15.37:0/984236730 >> v2:172.21.15.134:3300/0 conn(0x7f561815f410 0x7f56180b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.561+0000 7f561d904640 1 -- 172.21.15.37:0/984236730 >> 172.21.15.37:0/984236730 conn(0x7f561808ebb0 msgr2=0x7f561809e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:39.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.561+0000 7f561d904640 1 -- 172.21.15.37:0/984236730 shutdown_connections 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.561+0000 7f561d904640 1 -- 172.21.15.37:0/984236730 wait complete. 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.562+0000 7f561d904640 1 Processor -- start 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.562+0000 7f561d904640 1 -- start start 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f561d904640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56180d3e80 0x7f561815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f561d904640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f561815f410 0x7f561815ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f561d904640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56181f7f60 0x7f561815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f561d904640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56180b5240 con 0x7f561815f410 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f561d904640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56180b50c0 con 0x7f56180d3e80 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f561d904640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f56180d0650 con 0x7f56181f7f60 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f56167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f561815f410 0x7f561815ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f56167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f561815f410 0x7f561815ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48340/0 (socket says 172.21.15.37:48340) 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f56167fc640 1 -- 172.21.15.37:0/2097286836 learned_addr learned my addr 172.21.15.37:0/2097286836 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f56167fc640 1 -- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56181f7f60 msgr2=0x7f561815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f56177fe640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56181f7f60 0x7f561815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f56167fc640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56181f7f60 0x7f561815c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f56167fc640 1 -- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56180d3e80 msgr2=0x7f561815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f56167fc640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56180d3e80 0x7f561815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f56167fc640 1 -- 172.21.15.37:0/2097286836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56181f8940 con 0x7f561815f410 2024-04-26T20:22:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f56167fc640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f561815f410 0x7f561815ff10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c430 tx=0x7f560c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f55fffff640 1 -- 172.21.15.37:0/2097286836 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c0074e0 con 0x7f561815f410 2024-04-26T20:22:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f55fffff640 1 -- 172.21.15.37:0/2097286836 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f560c0047e0 con 0x7f561815f410 2024-04-26T20:22:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f55fffff640 1 -- 172.21.15.37:0/2097286836 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c004bd0 con 0x7f561815f410 2024-04-26T20:22:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.563+0000 7f561d904640 1 -- 172.21.15.37:0/2097286836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56181f8c20 con 0x7f561815f410 2024-04-26T20:22:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.564+0000 7f561d904640 1 -- 172.21.15.37:0/2097286836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56180ce1d0 con 0x7f561815f410 2024-04-26T20:22:39.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.565+0000 7f561d904640 1 -- 172.21.15.37:0/2097286836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56180ce3d0 con 0x7f561815f410 2024-04-26T20:22:39.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.565+0000 7f55fffff640 1 -- 172.21.15.37:0/2097286836 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f560c011070 con 0x7f561815f410 2024-04-26T20:22:39.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.566+0000 7f55fffff640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f560406c920 0x7f560406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.566+0000 7f55fffff640 1 -- 172.21.15.37:0/2097286836 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f560c08b380 con 0x7f561815f410 2024-04-26T20:22:39.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.566+0000 7f5616ffd640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f560406c920 0x7f560406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.566+0000 7f5616ffd640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f560406c920 0x7f560406ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5608006fd0 tx=0x7f5608008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:39.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.570+0000 7f55fffff640 1 -- 172.21.15.37:0/2097286836 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f56180ce3d0 con 0x7f561815f410 2024-04-26T20:22:39.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:39.758+0000 7f561d904640 1 -- 172.21.15.37:0/2097286836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f56180d0b70 con 0x7f561815f410 2024-04-26T20:22:40.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.729+0000 7f55fffff640 1 -- 172.21.15.37:0/2097286836 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f560c059b90 con 0x7f561815f410 2024-04-26T20:22:40.730 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-26T20:22:40.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 -- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f560406c920 msgr2=0x7f560406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:40.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f560406c920 0x7f560406ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5608006fd0 tx=0x7f5608008040 comp rx=0 tx=0).stop 2024-04-26T20:22:40.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 -- 172.21.15.37:0/2097286836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f561815f410 msgr2=0x7f561815ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:40.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f561815f410 0x7f561815ff10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c430 tx=0x7f560c00c8f0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 -- 172.21.15.37:0/2097286836 shutdown_connections 2024-04-26T20:22:40.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f560406c920 0x7f560406ede0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56181f7f60 0x7f561815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f561815f410 0x7f561815ff10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 --2- 172.21.15.37:0/2097286836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56180d3e80 0x7f561815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 -- 172.21.15.37:0/2097286836 >> 172.21.15.37:0/2097286836 conn(0x7f561808ebb0 msgr2=0x7f56180b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:40.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 -- 172.21.15.37:0/2097286836 shutdown_connections 2024-04-26T20:22:40.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.732+0000 7f561d904640 1 -- 172.21.15.37:0/2097286836 wait complete. 2024-04-26T20:22:40.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:22:40.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.864+0000 7f6813df0640 1 Processor -- start 2024-04-26T20:22:40.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.864+0000 7f6813df0640 1 -- start start 2024-04-26T20:22:40.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f6813df0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f680c15f4b0 0x7f680c0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:40.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f6813df0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f680c0b40d0 0x7f680c0b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:40.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f6813df0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f680c0b4540 0x7f680c0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:40.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f6813df0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f680c0b3d20 con 0x7f680c0b40d0 2024-04-26T20:22:40.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f6813df0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f680c0ae0b0 con 0x7f680c15f4b0 2024-04-26T20:22:40.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f6813df0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f680c0ae390 con 0x7f680c0b4540 2024-04-26T20:22:40.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f681138b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f680c0b40d0 0x7f680c0b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:40.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f681138b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f680c0b40d0 0x7f680c0b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48352/0 (socket says 172.21.15.37:48352) 2024-04-26T20:22:40.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f681138b640 1 -- 172.21.15.37:0/3699308248 learned_addr learned my addr 172.21.15.37:0/3699308248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:40.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f6811b8c640 1 --2- 172.21.15.37:0/3699308248 >> v2:172.21.15.134:3300/0 conn(0x7f680c15f4b0 0x7f680c0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:40.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f681238d640 1 --2- 172.21.15.37:0/3699308248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f680c0b4540 0x7f680c0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:40.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f681138b640 1 -- 172.21.15.37:0/3699308248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f680c0b4540 msgr2=0x7f680c0b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:40.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f681138b640 1 --2- 172.21.15.37:0/3699308248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f680c0b4540 0x7f680c0b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f681138b640 1 -- 172.21.15.37:0/3699308248 >> v2:172.21.15.134:3300/0 conn(0x7f680c15f4b0 msgr2=0x7f680c0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:40.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f681138b640 1 --2- 172.21.15.37:0/3699308248 >> v2:172.21.15.134:3300/0 conn(0x7f680c15f4b0 0x7f680c0b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.865+0000 7f681138b640 1 -- 172.21.15.37:0/3699308248 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c0ae670 con 0x7f680c0b40d0 2024-04-26T20:22:40.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.866+0000 7f681138b640 1 --2- 172.21.15.37:0/3699308248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c0b40d0 0x7f680c0b16b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f680c073600 tx=0x7f67fc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750c47a41e3761b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:40.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.866+0000 7f6810b8a640 1 -- 172.21.15.37:0/3699308248 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc037070 con 0x7f680c0b40d0 2024-04-26T20:22:40.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.866+0000 7f6810b8a640 1 -- 172.21.15.37:0/3699308248 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67fc02ae00 con 0x7f680c0b40d0 2024-04-26T20:22:40.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.866+0000 7f6810b8a640 1 -- 172.21.15.37:0/3699308248 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc032660 con 0x7f680c0b40d0 2024-04-26T20:22:40.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.867+0000 7f6813df0640 1 -- 172.21.15.37:0/3699308248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c0b40d0 msgr2=0x7f680c0b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:40.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.867+0000 7f6813df0640 1 --2- 172.21.15.37:0/3699308248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c0b40d0 0x7f680c0b16b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f680c073600 tx=0x7f67fc02a0b0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.867+0000 7f6813df0640 1 -- 172.21.15.37:0/3699308248 shutdown_connections 2024-04-26T20:22:40.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.867+0000 7f6813df0640 1 --2- 172.21.15.37:0/3699308248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f680c0b4540 0x7f680c0b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.867+0000 7f6813df0640 1 --2- 172.21.15.37:0/3699308248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c0b40d0 0x7f680c0b16b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.867+0000 7f6813df0640 1 --2- 172.21.15.37:0/3699308248 >> v2:172.21.15.134:3300/0 conn(0x7f680c15f4b0 0x7f680c0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.867+0000 7f6813df0640 1 -- 172.21.15.37:0/3699308248 >> 172.21.15.37:0/3699308248 conn(0x7f680c0b6630 msgr2=0x7f680c0b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:40.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.867+0000 7f6813df0640 1 -- 172.21.15.37:0/3699308248 shutdown_connections 2024-04-26T20:22:40.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.867+0000 7f6813df0640 1 -- 172.21.15.37:0/3699308248 wait complete. 2024-04-26T20:22:40.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.868+0000 7f6813df0640 1 Processor -- start 2024-04-26T20:22:40.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.868+0000 7f6813df0640 1 -- start start 2024-04-26T20:22:40.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6813df0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c0b40d0 0x7f680c154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:40.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6813df0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f680c0b4540 0x7f680c161d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:40.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6813df0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f680c15f4b0 0x7f680c0d1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:40.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6813df0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f680c0b0cd0 con 0x7f680c0b40d0 2024-04-26T20:22:40.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6813df0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f680c0aec30 con 0x7f680c15f4b0 2024-04-26T20:22:40.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6813df0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f680c0b0e50 con 0x7f680c0b4540 2024-04-26T20:22:40.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6811b8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c0b40d0 0x7f680c154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:40.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6811b8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c0b40d0 0x7f680c154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48360/0 (socket says 172.21.15.37:48360) 2024-04-26T20:22:40.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6811b8c640 1 -- 172.21.15.37:0/632872136 learned_addr learned my addr 172.21.15.37:0/632872136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:40.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f681138b640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f680c0b4540 0x7f680c161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:40.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f681238d640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f680c15f4b0 0x7f680c0d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:40.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6811b8c640 1 -- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f680c0b4540 msgr2=0x7f680c161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:40.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6811b8c640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f680c0b4540 0x7f680c161d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6811b8c640 1 -- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f680c15f4b0 msgr2=0x7f680c0d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:40.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6811b8c640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f680c15f4b0 0x7f680c0d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6811b8c640 1 -- 172.21.15.37:0/632872136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c1623d0 con 0x7f680c0b40d0 2024-04-26T20:22:40.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f681238d640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f680c15f4b0 0x7f680c0d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:40.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.869+0000 7f6811b8c640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c0b40d0 0x7f680c154930 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f680800e8a0 tx=0x7f680800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:40.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.870+0000 7f68027fc640 1 -- 172.21.15.37:0/632872136 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808017070 con 0x7f680c0b40d0 2024-04-26T20:22:40.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.870+0000 7f68027fc640 1 -- 172.21.15.37:0/632872136 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f680800c940 con 0x7f680c0b40d0 2024-04-26T20:22:40.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.870+0000 7f68027fc640 1 -- 172.21.15.37:0/632872136 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680800cd30 con 0x7f680c0b40d0 2024-04-26T20:22:40.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.870+0000 7f6813df0640 1 -- 172.21.15.37:0/632872136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680c06e680 con 0x7f680c0b40d0 2024-04-26T20:22:40.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.870+0000 7f6813df0640 1 -- 172.21.15.37:0/632872136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680c1f70a0 con 0x7f680c0b40d0 2024-04-26T20:22:40.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.870+0000 7f6813df0640 1 -- 172.21.15.37:0/632872136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f680c06a360 con 0x7f680c0b40d0 2024-04-26T20:22:40.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.871+0000 7f68027fc640 1 -- 172.21.15.37:0/632872136 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f680801a070 con 0x7f680c0b40d0 2024-04-26T20:22:40.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.872+0000 7f68027fc640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f67f006c920 0x7f67f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:40.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.872+0000 7f68027fc640 1 -- 172.21.15.37:0/632872136 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f680808b860 con 0x7f680c0b40d0 2024-04-26T20:22:40.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.872+0000 7f681138b640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f67f006c920 0x7f67f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:40.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.872+0000 7f681138b640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f67f006c920 0x7f67f006ede0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f67fc034000 tx=0x7f67fc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:40.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:40.876+0000 7f68027fc640 1 -- 172.21.15.37:0/632872136 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6808059790 con 0x7f680c0b40d0 2024-04-26T20:22:41.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.061+0000 7f6813df0640 1 -- 172.21.15.37:0/632872136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f680c15db50 con 0x7f680c0b40d0 2024-04-26T20:22:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.736+0000 7f68027fc640 1 -- 172.21.15.37:0/632872136 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f680805da90 con 0x7f680c0b40d0 2024-04-26T20:22:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-26T20:22:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.739+0000 7f6813df0640 1 -- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f67f006c920 msgr2=0x7f67f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.739+0000 7f6813df0640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f67f006c920 0x7f67f006ede0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f67fc034000 tx=0x7f67fc02e000 comp rx=0 tx=0).stop 2024-04-26T20:22:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.739+0000 7f6813df0640 1 -- 172.21.15.37:0/632872136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c0b40d0 msgr2=0x7f680c154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.739+0000 7f6813df0640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c0b40d0 0x7f680c154930 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f680800e8a0 tx=0x7f680800ed60 comp rx=0 tx=0).stop 2024-04-26T20:22:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.740+0000 7f6813df0640 1 -- 172.21.15.37:0/632872136 shutdown_connections 2024-04-26T20:22:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.740+0000 7f6813df0640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f680c15f4b0 0x7f680c0d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.740+0000 7f6813df0640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f67f006c920 0x7f67f006ede0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.740+0000 7f6813df0640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f680c0b4540 0x7f680c161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.740+0000 7f6813df0640 1 --2- 172.21.15.37:0/632872136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c0b40d0 0x7f680c154930 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.740+0000 7f6813df0640 1 -- 172.21.15.37:0/632872136 >> 172.21.15.37:0/632872136 conn(0x7f680c0b6630 msgr2=0x7f680c0af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.740+0000 7f6813df0640 1 -- 172.21.15.37:0/632872136 shutdown_connections 2024-04-26T20:22:41.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.740+0000 7f6813df0640 1 -- 172.21.15.37:0/632872136 wait complete. 2024-04-26T20:22:41.768 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:22:41.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.875+0000 7f4626fb4640 1 Processor -- start 2024-04-26T20:22:41.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.875+0000 7f4626fb4640 1 -- start start 2024-04-26T20:22:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f4626fb4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f462015d630 0x7f46200b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f4626fb4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46200d0dd0 0x7f46200b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f4626fb4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46200d09d0 con 0x7f46200d15d0 2024-04-26T20:22:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f4626fb4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f46200cf500 con 0x7f462015d630 2024-04-26T20:22:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f4626fb4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f46200cec60 con 0x7f46200d0dd0 2024-04-26T20:22:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f461ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f462015d630 0x7f46200b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f461ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f462015d630 0x7f46200b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48370/0 (socket says 172.21.15.37:48370) 2024-04-26T20:22:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f461ffff640 1 -- 172.21.15.37:0/4181610118 learned_addr learned my addr 172.21.15.37:0/4181610118 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f4625551640 1 --2- 172.21.15.37:0/4181610118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46200d0dd0 0x7f46200b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:41.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f461ffff640 1 -- 172.21.15.37:0/4181610118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46200d0dd0 msgr2=0x7f46200b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:41.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f461ffff640 1 --2- 172.21.15.37:0/4181610118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46200d0dd0 0x7f46200b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:41.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f461ffff640 1 -- 172.21.15.37:0/4181610118 >> v1:172.21.15.134:6789/0 conn(0x7f46200d15d0 legacy=0x7f46200d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:41.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.876+0000 7f461ffff640 1 -- 172.21.15.37:0/4181610118 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46200cef40 con 0x7f462015d630 2024-04-26T20:22:41.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.877+0000 7f461ffff640 1 --2- 172.21.15.37:0/4181610118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462015d630 0x7f46200b2a90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f460c009870 tx=0x7f460c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afb30e636a42e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:41.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.877+0000 7f461f7fe640 1 -- 172.21.15.37:0/4181610118 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c037070 con 0x7f462015d630 2024-04-26T20:22:41.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.877+0000 7f461f7fe640 1 -- 172.21.15.37:0/4181610118 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460c032440 con 0x7f462015d630 2024-04-26T20:22:41.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.877+0000 7f461f7fe640 1 -- 172.21.15.37:0/4181610118 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c032810 con 0x7f462015d630 2024-04-26T20:22:41.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.878+0000 7f4626fb4640 1 -- 172.21.15.37:0/4181610118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462015d630 msgr2=0x7f46200b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:41.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.878+0000 7f4626fb4640 1 --2- 172.21.15.37:0/4181610118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462015d630 0x7f46200b2a90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f460c009870 tx=0x7f460c02a270 comp rx=0 tx=0).stop 2024-04-26T20:22:41.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.878+0000 7f4626fb4640 1 -- 172.21.15.37:0/4181610118 shutdown_connections 2024-04-26T20:22:41.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.878+0000 7f4626fb4640 1 --2- 172.21.15.37:0/4181610118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46200d0dd0 0x7f46200b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:41.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.878+0000 7f4626fb4640 1 --2- 172.21.15.37:0/4181610118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462015d630 0x7f46200b2a90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:41.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.878+0000 7f4626fb4640 1 -- 172.21.15.37:0/4181610118 >> 172.21.15.37:0/4181610118 conn(0x7f46200d2ff0 msgr2=0x7f46200d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:41.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.878+0000 7f4626fb4640 1 -- 172.21.15.37:0/4181610118 shutdown_connections 2024-04-26T20:22:41.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.878+0000 7f4626fb4640 1 -- 172.21.15.37:0/4181610118 wait complete. 2024-04-26T20:22:41.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.879+0000 7f4626fb4640 1 Processor -- start 2024-04-26T20:22:41.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.879+0000 7f4626fb4640 1 -- start start 2024-04-26T20:22:41.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f4626fb4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46200d0dd0 0x7f46200b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:41.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f4626fb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46200d15d0 0x7f462014c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:41.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f4626fb4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f462015d630 0x7f46200b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:41.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f4626fb4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46200ce8f0 con 0x7f46200d15d0 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f4626fb4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46200b2570 con 0x7f46200d0dd0 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f4626fb4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f46200c67a0 con 0x7f462015d630 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f461ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46200d15d0 0x7f462014c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f461ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46200d15d0 0x7f462014c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48380/0 (socket says 172.21.15.37:48380) 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f461ffff640 1 -- 172.21.15.37:0/989041506 learned_addr learned my addr 172.21.15.37:0/989041506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f4625551640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f462015d630 0x7f46200b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f4624d50640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46200d0dd0 0x7f46200b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f461ffff640 1 -- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f462015d630 msgr2=0x7f46200b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f461ffff640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f462015d630 0x7f46200b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f461ffff640 1 -- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46200d0dd0 msgr2=0x7f46200b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f461ffff640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46200d0dd0 0x7f46200b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f461ffff640 1 -- 172.21.15.37:0/989041506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46200b6090 con 0x7f46200d15d0 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f4625551640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f462015d630 0x7f46200b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f461ffff640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46200d15d0 0x7f462014c390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f460c000c00 tx=0x7f460c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:41.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.880+0000 7f461d7fa640 1 -- 172.21.15.37:0/989041506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c037040 con 0x7f46200d15d0 2024-04-26T20:22:41.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.881+0000 7f461d7fa640 1 -- 172.21.15.37:0/989041506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460c0304c0 con 0x7f46200d15d0 2024-04-26T20:22:41.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.881+0000 7f4626fb4640 1 -- 172.21.15.37:0/989041506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46200b6370 con 0x7f46200d15d0 2024-04-26T20:22:41.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.881+0000 7f461d7fa640 1 -- 172.21.15.37:0/989041506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c0308b0 con 0x7f46200d15d0 2024-04-26T20:22:41.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.881+0000 7f4626fb4640 1 -- 172.21.15.37:0/989041506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46200b6930 con 0x7f46200d15d0 2024-04-26T20:22:41.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.882+0000 7f4626fb4640 1 -- 172.21.15.37:0/989041506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462006a360 con 0x7f46200d15d0 2024-04-26T20:22:41.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.882+0000 7f461d7fa640 1 -- 172.21.15.37:0/989041506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f460c044020 con 0x7f46200d15d0 2024-04-26T20:22:41.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.883+0000 7f461d7fa640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f460406c8f0 0x7f460406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:41.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.883+0000 7f461d7fa640 1 -- 172.21.15.37:0/989041506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f460c0abb90 con 0x7f46200d15d0 2024-04-26T20:22:41.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.883+0000 7f4624d50640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f460406c8f0 0x7f460406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:41.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.883+0000 7f4624d50640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f460406c8f0 0x7f460406edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4614001640 tx=0x7f4614009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:41.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:41.887+0000 7f461d7fa640 1 -- 172.21.15.37:0/989041506 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f460c04b020 con 0x7f46200d15d0 2024-04-26T20:22:42.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.115+0000 7f4626fb4640 1 -- 172.21.15.37:0/989041506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f46200744b0 con 0x7f46200d15d0 2024-04-26T20:22:42.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.738+0000 7f461d7fa640 1 -- 172.21.15.37:0/989041506 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f460c07a020 con 0x7f46200d15d0 2024-04-26T20:22:42.739 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 4 and data pool 5 2024-04-26T20:22:42.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.746+0000 7f4626fb4640 1 -- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f460406c8f0 msgr2=0x7f460406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:42.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.746+0000 7f4626fb4640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f460406c8f0 0x7f460406edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4614001640 tx=0x7f4614009040 comp rx=0 tx=0).stop 2024-04-26T20:22:42.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.746+0000 7f4626fb4640 1 -- 172.21.15.37:0/989041506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46200d15d0 msgr2=0x7f462014c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:42.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.746+0000 7f4626fb4640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46200d15d0 0x7f462014c390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f460c000c00 tx=0x7f460c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.748+0000 7f4626fb4640 1 -- 172.21.15.37:0/989041506 shutdown_connections 2024-04-26T20:22:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.748+0000 7f4626fb4640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f460406c8f0 0x7f460406edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.748+0000 7f4626fb4640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f462015d630 0x7f46200b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.748+0000 7f4626fb4640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46200d15d0 0x7f462014c390 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:42.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.748+0000 7f4626fb4640 1 --2- 172.21.15.37:0/989041506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46200d0dd0 0x7f46200b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:42.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.748+0000 7f4626fb4640 1 -- 172.21.15.37:0/989041506 >> 172.21.15.37:0/989041506 conn(0x7f46200d2ff0 msgr2=0x7f46200b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:42.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.750+0000 7f4626fb4640 1 -- 172.21.15.37:0/989041506 shutdown_connections 2024-04-26T20:22:42.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.751+0000 7f4626fb4640 1 -- 172.21.15.37:0/989041506 wait complete. 2024-04-26T20:22:42.780 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:42.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.882+0000 7f3209b2c640 1 Processor -- start 2024-04-26T20:22:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.883+0000 7f3209b2c640 1 -- start start 2024-04-26T20:22:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.883+0000 7f3209b2c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32040d3820 0x7f32040d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.883+0000 7f3209b2c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f32040b4ba0 0x7f32040b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.883+0000 7f3209b2c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32040d0210 con 0x7f3204162b40 2024-04-26T20:22:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.883+0000 7f3209b2c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f32040d41c0 con 0x7f32040b4ba0 2024-04-26T20:22:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.883+0000 7f3209b2c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f32040d04f0 con 0x7f32040d3820 2024-04-26T20:22:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.884+0000 7f3202ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32040d3820 0x7f32040d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.884+0000 7f3203fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f32040b4ba0 0x7f32040b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.884+0000 7f32037fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f3204162b40 0x7f32040d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:42584/0 (socket says 172.21.15.37:42584) 2024-04-26T20:22:42.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.884+0000 7f32037fe640 1 -- 172.21.15.37:0/598886747 learned_addr learned my addr 172.21.15.37:0/598886747 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.884+0000 7f32027fc640 1 -- 172.21.15.37:0/598886747 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3102452032 0 0) 0x7f32040d0210 con 0x7f3204162b40 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.884+0000 7f32027fc640 1 -- 172.21.15.37:0/598886747 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31ec003520 con 0x7f3204162b40 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.884+0000 7f3202ffd640 1 -- 172.21.15.37:0/598886747 >> v2:172.21.15.134:3300/0 conn(0x7f32040b4ba0 msgr2=0x7f32040b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.884+0000 7f3202ffd640 1 --2- 172.21.15.37:0/598886747 >> v2:172.21.15.134:3300/0 conn(0x7f32040b4ba0 0x7f32040b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.884+0000 7f3202ffd640 1 -- 172.21.15.37:0/598886747 >> v1:172.21.15.134:6789/0 conn(0x7f3204162b40 legacy=0x7f32040d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.884+0000 7f3202ffd640 1 -- 172.21.15.37:0/598886747 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32040d07d0 con 0x7f32040d3820 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.885+0000 7f3202ffd640 1 --2- 172.21.15.37:0/598886747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32040d3820 0x7f32040d3c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f31f80097a0 tx=0x7f31f802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89683aaa0e284ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.885+0000 7f32027fc640 1 -- 172.21.15.37:0/598886747 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f8037070 con 0x7f32040d3820 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.885+0000 7f32027fc640 1 -- 172.21.15.37:0/598886747 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f8032440 con 0x7f32040d3820 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.885+0000 7f32027fc640 1 -- 172.21.15.37:0/598886747 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f8032810 con 0x7f32040d3820 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.885+0000 7f3209b2c640 1 -- 172.21.15.37:0/598886747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32040d3820 msgr2=0x7f32040d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.885+0000 7f3209b2c640 1 --2- 172.21.15.37:0/598886747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32040d3820 0x7f32040d3c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f31f80097a0 tx=0x7f31f802a180 comp rx=0 tx=0).stop 2024-04-26T20:22:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.885+0000 7f3209b2c640 1 -- 172.21.15.37:0/598886747 shutdown_connections 2024-04-26T20:22:42.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.885+0000 7f3209b2c640 1 --2- 172.21.15.37:0/598886747 >> v2:172.21.15.134:3300/0 conn(0x7f32040b4ba0 0x7f32040b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:42.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.885+0000 7f3209b2c640 1 --2- 172.21.15.37:0/598886747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32040d3820 0x7f32040d3c80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:42.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.885+0000 7f3209b2c640 1 -- 172.21.15.37:0/598886747 >> 172.21.15.37:0/598886747 conn(0x7f320409e550 msgr2=0x7f32040d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:42.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.885+0000 7f3209b2c640 1 -- 172.21.15.37:0/598886747 shutdown_connections 2024-04-26T20:22:42.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.886+0000 7f3209b2c640 1 -- 172.21.15.37:0/598886747 wait complete. 2024-04-26T20:22:42.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.887+0000 7f3209b2c640 1 Processor -- start 2024-04-26T20:22:42.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.887+0000 7f3209b2c640 1 -- start start 2024-04-26T20:22:42.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.887+0000 7f3209b2c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32040b4ba0 0x7f320414ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:42.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.887+0000 7f3209b2c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32040d3820 0x7f320414ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:42.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.887+0000 7f3209b2c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3204162b40 0x7f32041f9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:42.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.887+0000 7f3209b2c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f32040d12b0 con 0x7f32040b4ba0 2024-04-26T20:22:42.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.887+0000 7f3209b2c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f32040d0d90 con 0x7f32040d3820 2024-04-26T20:22:42.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.887+0000 7f3209b2c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f32040cfe30 con 0x7f3204162b40 2024-04-26T20:22:42.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f32037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32040b4ba0 0x7f320414ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:42.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f32037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32040b4ba0 0x7f320414ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48394/0 (socket says 172.21.15.37:48394) 2024-04-26T20:22:42.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f32037fe640 1 -- 172.21.15.37:0/2712721530 learned_addr learned my addr 172.21.15.37:0/2712721530 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f3202ffd640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32040d3820 0x7f320414ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f3203fff640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3204162b40 0x7f32041f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f32037fe640 1 -- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3204162b40 msgr2=0x7f32041f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f32037fe640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3204162b40 0x7f32041f9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f32037fe640 1 -- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32040d3820 msgr2=0x7f320414ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f32037fe640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32040d3820 0x7f320414ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f32037fe640 1 -- 172.21.15.37:0/2712721530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320414f0e0 con 0x7f32040b4ba0 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f32037fe640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32040b4ba0 0x7f320414ead0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c390 tx=0x7f31f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f31ebfff640 1 -- 172.21.15.37:0/2712721530 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4017070 con 0x7f32040b4ba0 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f31ebfff640 1 -- 172.21.15.37:0/2712721530 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f4005ce0 con 0x7f32040b4ba0 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f3209b2c640 1 -- 172.21.15.37:0/2712721530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320414f3c0 con 0x7f32040b4ba0 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f31ebfff640 1 -- 172.21.15.37:0/2712721530 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f40072f0 con 0x7f32040b4ba0 2024-04-26T20:22:42.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.888+0000 7f3209b2c640 1 -- 172.21.15.37:0/2712721530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320414f980 con 0x7f32040b4ba0 2024-04-26T20:22:42.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.889+0000 7f3209b2c640 1 -- 172.21.15.37:0/2712721530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f320406a360 con 0x7f32040b4ba0 2024-04-26T20:22:42.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.890+0000 7f31ebfff640 1 -- 172.21.15.37:0/2712721530 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31f401a070 con 0x7f32040b4ba0 2024-04-26T20:22:42.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.890+0000 7f31ebfff640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f31f006c8f0 0x7f31f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:42.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.890+0000 7f31ebfff640 1 -- 172.21.15.37:0/2712721530 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f31f408af40 con 0x7f32040b4ba0 2024-04-26T20:22:42.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.890+0000 7f3202ffd640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f31f006c8f0 0x7f31f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:42.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.891+0000 7f3202ffd640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f31f006c8f0 0x7f31f006edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f31f8034000 tx=0x7f31f802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:42.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:42.895+0000 7f31ebfff640 1 -- 172.21.15.37:0/2712721530 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f31f4058ca0 con 0x7f32040b4ba0 2024-04-26T20:22:43.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.080+0000 7f3209b2c640 1 -- 172.21.15.37:0/2712721530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32040d2960 con 0x7f32040b4ba0 2024-04-26T20:22:43.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.081+0000 7f31ebfff640 1 -- 172.21.15.37:0/2712721530 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16694 (secure 0 0 0) 0x7f31f405cfa0 con 0x7f32040b4ba0 2024-04-26T20:22:43.082 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:43.082 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":32,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:22:42.116946+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:22:39.762703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:22:41.066360+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:43.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 -- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f31f006c8f0 msgr2=0x7f31f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f31f006c8f0 0x7f31f006edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f31f8034000 tx=0x7f31f802e000 comp rx=0 tx=0).stop 2024-04-26T20:22:43.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 -- 172.21.15.37:0/2712721530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32040b4ba0 msgr2=0x7f320414ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32040b4ba0 0x7f320414ead0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c390 tx=0x7f31f400c850 comp rx=0 tx=0).stop 2024-04-26T20:22:43.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 -- 172.21.15.37:0/2712721530 shutdown_connections 2024-04-26T20:22:43.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f31f006c8f0 0x7f31f006edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3204162b40 0x7f32041f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32040d3820 0x7f320414ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 --2- 172.21.15.37:0/2712721530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32040b4ba0 0x7f320414ead0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 -- 172.21.15.37:0/2712721530 >> 172.21.15.37:0/2712721530 conn(0x7f320409e550 msgr2=0x7f320406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:43.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 -- 172.21.15.37:0/2712721530 shutdown_connections 2024-04-26T20:22:43.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.083+0000 7f3209b2c640 1 -- 172.21.15.37:0/2712721530 wait complete. 2024-04-26T20:22:43.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:43.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.195+0000 7f11cd56a640 1 Processor -- start 2024-04-26T20:22:43.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.195+0000 7f11cd56a640 1 -- start start 2024-04-26T20:22:43.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11cd56a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f11c8160a00 0x7f11c80b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11cd56a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 0x7f11c80b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11cd56a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11c80cf740 con 0x7f11c80cf9c0 2024-04-26T20:22:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11cd56a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f11c80cf080 con 0x7f11c8160a00 2024-04-26T20:22:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11cd56a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f11c80ce7e0 con 0x7f11c80d0540 2024-04-26T20:22:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11c6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f11c8160a00 0x7f11c80b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11c6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f11c8160a00 0x7f11c80b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48398/0 (socket says 172.21.15.37:48398) 2024-04-26T20:22:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11c6ffd640 1 -- 172.21.15.37:0/78160175 learned_addr learned my addr 172.21.15.37:0/78160175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11c6ffd640 1 -- 172.21.15.37:0/78160175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 msgr2=0x7f11c80b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11c77fe640 1 --2- 172.21.15.37:0/78160175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 0x7f11c80b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11c6ffd640 1 --2- 172.21.15.37:0/78160175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 0x7f11c80b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11c6ffd640 1 -- 172.21.15.37:0/78160175 >> v1:172.21.15.134:6789/0 conn(0x7f11c80cf9c0 legacy=0x7f11c80cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11c6ffd640 1 -- 172.21.15.37:0/78160175 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c80ceac0 con 0x7f11c8160a00 2024-04-26T20:22:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.196+0000 7f11c77fe640 1 --2- 172.21.15.37:0/78160175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 0x7f11c80b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.197+0000 7f11c6ffd640 1 --2- 172.21.15.37:0/78160175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c8160a00 0x7f11c80b1f90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009870 tx=0x7f11b802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7a7aef3c51e1b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.197+0000 7f11c5ffb640 1 -- 172.21.15.37:0/78160175 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8037070 con 0x7f11c8160a00 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.197+0000 7f11c5ffb640 1 -- 172.21.15.37:0/78160175 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b8032440 con 0x7f11c8160a00 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.197+0000 7f11c5ffb640 1 -- 172.21.15.37:0/78160175 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8032810 con 0x7f11c8160a00 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.198+0000 7f11cd56a640 1 -- 172.21.15.37:0/78160175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c8160a00 msgr2=0x7f11c80b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.198+0000 7f11cd56a640 1 --2- 172.21.15.37:0/78160175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c8160a00 0x7f11c80b1f90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009870 tx=0x7f11b802a2a0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.198+0000 7f11cd56a640 1 -- 172.21.15.37:0/78160175 shutdown_connections 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.198+0000 7f11cd56a640 1 --2- 172.21.15.37:0/78160175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 0x7f11c80b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.198+0000 7f11cd56a640 1 --2- 172.21.15.37:0/78160175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c8160a00 0x7f11c80b1f90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.198+0000 7f11cd56a640 1 -- 172.21.15.37:0/78160175 >> 172.21.15.37:0/78160175 conn(0x7f11c809ba50 msgr2=0x7f11c809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.198+0000 7f11cd56a640 1 -- 172.21.15.37:0/78160175 shutdown_connections 2024-04-26T20:22:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.198+0000 7f11cd56a640 1 -- 172.21.15.37:0/78160175 wait complete. 2024-04-26T20:22:43.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.199+0000 7f11cd56a640 1 Processor -- start 2024-04-26T20:22:43.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.199+0000 7f11cd56a640 1 -- start start 2024-04-26T20:22:43.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11cd56a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c80cf9c0 0x7f11c81f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11cd56a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 0x7f11c80cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11cd56a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c8160a00 0x7f11c80b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11cd56a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11c80b2690 con 0x7f11c80cf9c0 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11cd56a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11c8160de0 con 0x7f11c8160a00 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11cd56a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f11c80cc530 con 0x7f11c80d0540 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c80cf9c0 0x7f11c81f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c80cf9c0 0x7f11c81f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48404/0 (socket says 172.21.15.37:48404) 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c6ffd640 1 -- 172.21.15.37:0/3559759015 learned_addr learned my addr 172.21.15.37:0/3559759015 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c77fe640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c8160a00 0x7f11c80b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c67fc640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 0x7f11c80cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c6ffd640 1 -- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 msgr2=0x7f11c80cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c6ffd640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 0x7f11c80cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c6ffd640 1 -- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c8160a00 msgr2=0x7f11c80b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c6ffd640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c8160a00 0x7f11c80b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c6ffd640 1 -- 172.21.15.37:0/3559759015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c81f8500 con 0x7f11c80cf9c0 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c77fe640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c8160a00 0x7f11c80b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:43.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c67fc640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 0x7f11c80cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:43.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11c6ffd640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c80cf9c0 0x7f11c81f8fe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009870 tx=0x7f11b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:43.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11af7fe640 1 -- 172.21.15.37:0/3559759015 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8041020 con 0x7f11c80cf9c0 2024-04-26T20:22:43.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11af7fe640 1 -- 172.21.15.37:0/3559759015 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b8030970 con 0x7f11c80cf9c0 2024-04-26T20:22:43.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11cd56a640 1 -- 172.21.15.37:0/3559759015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c81f87e0 con 0x7f11c80cf9c0 2024-04-26T20:22:43.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.200+0000 7f11af7fe640 1 -- 172.21.15.37:0/3559759015 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8037040 con 0x7f11c80cf9c0 2024-04-26T20:22:43.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.201+0000 7f11cd56a640 1 -- 172.21.15.37:0/3559759015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c80cc170 con 0x7f11c80cf9c0 2024-04-26T20:22:43.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.201+0000 7f11cd56a640 1 -- 172.21.15.37:0/3559759015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c81f89b0 con 0x7f11c80cf9c0 2024-04-26T20:22:43.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.202+0000 7f11af7fe640 1 -- 172.21.15.37:0/3559759015 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11b8044020 con 0x7f11c80cf9c0 2024-04-26T20:22:43.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.202+0000 7f11af7fe640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f11b406c8f0 0x7f11b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.203+0000 7f11af7fe640 1 -- 172.21.15.37:0/3559759015 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f11b80ac540 con 0x7f11c80cf9c0 2024-04-26T20:22:43.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.203+0000 7f11c67fc640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f11b406c8f0 0x7f11b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.203+0000 7f11c67fc640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f11b406c8f0 0x7f11b406edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f11c80cdf20 tx=0x7f11bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:43.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.207+0000 7f11af7fe640 1 -- 172.21.15.37:0/3559759015 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f11b804b030 con 0x7f11c80cf9c0 2024-04-26T20:22:43.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.393+0000 7f11cd56a640 1 -- 172.21.15.37:0/3559759015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11c806a360 con 0x7f11c80cf9c0 2024-04-26T20:22:43.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.394+0000 7f11af7fe640 1 -- 172.21.15.37:0/3559759015 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16694 (secure 0 0 0) 0x7f11b807a970 con 0x7f11c80cf9c0 2024-04-26T20:22:43.396 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:43.396 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":32,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:22:42.116946+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:22:39.762703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:22:41.066360+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.396+0000 7f11cd56a640 1 -- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f11b406c8f0 msgr2=0x7f11b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.396+0000 7f11cd56a640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f11b406c8f0 0x7f11b406edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f11c80cdf20 tx=0x7f11bc009040 comp rx=0 tx=0).stop 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.396+0000 7f11cd56a640 1 -- 172.21.15.37:0/3559759015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c80cf9c0 msgr2=0x7f11c81f8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.396+0000 7f11cd56a640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c80cf9c0 0x7f11c81f8fe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009870 tx=0x7f11b80046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.396+0000 7f11cd56a640 1 -- 172.21.15.37:0/3559759015 shutdown_connections 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.396+0000 7f11cd56a640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c8160a00 0x7f11c80b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.396+0000 7f11cd56a640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f11b406c8f0 0x7f11b406edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.396+0000 7f11cd56a640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11c80d0540 0x7f11c80cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.396+0000 7f11cd56a640 1 --2- 172.21.15.37:0/3559759015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c80cf9c0 0x7f11c81f8fe0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.396+0000 7f11cd56a640 1 -- 172.21.15.37:0/3559759015 >> 172.21.15.37:0/3559759015 conn(0x7f11c809ba50 msgr2=0x7f11c80afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.396+0000 7f11cd56a640 1 -- 172.21.15.37:0/3559759015 shutdown_connections 2024-04-26T20:22:43.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.397+0000 7f11cd56a640 1 -- 172.21.15.37:0/3559759015 wait complete. 2024-04-26T20:22:43.410 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:22:43.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.512+0000 7f9046924640 1 Processor -- start 2024-04-26T20:22:43.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.512+0000 7f9046924640 1 -- start start 2024-04-26T20:22:43.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f9046924640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f90400b2930 0x7f90400d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f9046924640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90400cd640 con 0x7f90400b2290 2024-04-26T20:22:43.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f9046924640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90400cd920 con 0x7f90400cff80 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f9046924640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f90400cdc00 con 0x7f90400b2930 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f90400b2290 0x7f90400cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33108/0 (socket says 172.21.15.37:33108) 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903ffff640 1 -- 172.21.15.37:0/1838258209 learned_addr learned my addr 172.21.15.37:0/1838258209 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f9044ec1640 1 --2- 172.21.15.37:0/1838258209 >> v2:172.21.15.134:3300/0 conn(0x7f90400b2930 0x7f90400d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3329047986 0 0) 0x7f90400cd640 con 0x7f90400b2290 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9028003520 con 0x7f90400b2290 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3723626115 0 0) 0x7f9028003520 con 0x7f90400b2290 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 >> v2:172.21.15.134:3300/0 conn(0x7f90400b2930 msgr2=0x7f90400d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 --2- 172.21.15.37:0/1838258209 >> v2:172.21.15.134:3300/0 conn(0x7f90400b2930 0x7f90400d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 >> v1:172.21.15.134:6789/0 conn(0x7f90400cff80 legacy=0x7f90400d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90400cdee0 con 0x7f90400b2290 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f9030001670 con 0x7f90400b2290 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 >> v1:172.21.15.37:6789/0 conn(0x7f90400b2290 legacy=0x7f90400cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 --2- 172.21.15.37:0/1838258209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90280039b0 0x7f9028003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 --2- 172.21.15.37:0/1838258209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 0x7f9028004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 --2- 172.21.15.37:0/1838258209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9028005080 0x7f902800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f90400cdc00 con 0x7f90280043b0 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f902800bfe0 con 0x7f9028005080 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.513+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f902800c1e0 con 0x7f90280039b0 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.514+0000 7f903f7fe640 1 --2- 172.21.15.37:0/1838258209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 0x7f9028004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.514+0000 7f903ffff640 1 --2- 172.21.15.37:0/1838258209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90280039b0 0x7f9028003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.514+0000 7f9044ec1640 1 --2- 172.21.15.37:0/1838258209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9028005080 0x7f902800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.514+0000 7f903f7fe640 1 -- 172.21.15.37:0/1838258209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90280039b0 msgr2=0x7f9028003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.514+0000 7f903f7fe640 1 --2- 172.21.15.37:0/1838258209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90280039b0 0x7f9028003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.514+0000 7f903f7fe640 1 -- 172.21.15.37:0/1838258209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9028005080 msgr2=0x7f902800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.514+0000 7f903f7fe640 1 --2- 172.21.15.37:0/1838258209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9028005080 0x7f902800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.514+0000 7f903f7fe640 1 -- 172.21.15.37:0/1838258209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9028003520 con 0x7f90280043b0 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.514+0000 7f9044ec1640 1 -- 172.21.15.37:0/1838258209 reap_dead start 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.515+0000 7f903f7fe640 1 --2- 172.21.15.37:0/1838258209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 0x7f9028004880 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9034001860 tx=0x7f9028005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6688e633b18db8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.515+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034037070 con 0x7f90280043b0 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.515+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903402abe0 con 0x7f90280043b0 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.515+0000 7f903effd640 1 -- 172.21.15.37:0/1838258209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034001430 con 0x7f90280043b0 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.515+0000 7f9046924640 1 -- 172.21.15.37:0/1838258209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 msgr2=0x7f9028004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.515+0000 7f9046924640 1 --2- 172.21.15.37:0/1838258209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 0x7f9028004880 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9034001860 tx=0x7f9028005830 comp rx=0 tx=0).stop 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.515+0000 7f9046924640 1 -- 172.21.15.37:0/1838258209 shutdown_connections 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.515+0000 7f9046924640 1 --2- 172.21.15.37:0/1838258209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 0x7f9028004880 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.515+0000 7f9046924640 1 -- 172.21.15.37:0/1838258209 >> 172.21.15.37:0/1838258209 conn(0x7f90400d2ff0 msgr2=0x7f90400d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.515+0000 7f9046924640 1 -- 172.21.15.37:0/1838258209 shutdown_connections 2024-04-26T20:22:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.515+0000 7f9046924640 1 -- 172.21.15.37:0/1838258209 wait complete. 2024-04-26T20:22:43.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.516+0000 7f9046924640 1 Processor -- start 2024-04-26T20:22:43.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.517+0000 7f9046924640 1 -- start start 2024-04-26T20:22:43.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.517+0000 7f9046924640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 0x7f90401f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.517+0000 7f9046924640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90401590a0 0x7f90400cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.517+0000 7f9046924640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9040159520 0x7f90401609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.517+0000 7f9046924640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f90400cf870 con 0x7f90280043b0 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.517+0000 7f9046924640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90400b2670 con 0x7f90401590a0 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.517+0000 7f9046924640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f90400cf9f0 con 0x7f9040159520 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 0x7f90401f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 0x7f90401f5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48420/0 (socket says 172.21.15.37:48420) 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903ffff640 1 -- 172.21.15.37:0/3721710516 learned_addr learned my addr 172.21.15.37:0/3721710516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903f7fe640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90401590a0 0x7f90400cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f9044ec1640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9040159520 0x7f90401609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903ffff640 1 -- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9040159520 msgr2=0x7f90401609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903ffff640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9040159520 0x7f90401609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903ffff640 1 -- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90401590a0 msgr2=0x7f90400cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903ffff640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90401590a0 0x7f90400cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903ffff640 1 -- 172.21.15.37:0/3721710516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90400b6150 con 0x7f90280043b0 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f9044ec1640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9040159520 0x7f90401609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903ffff640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 0x7f90401f5ca0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9030007310 tx=0x7f90300077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903cff9640 1 -- 172.21.15.37:0/3721710516 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903000f940 con 0x7f90280043b0 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903cff9640 1 -- 172.21.15.37:0/3721710516 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9030018070 con 0x7f90280043b0 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f9046924640 1 -- 172.21.15.37:0/3721710516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90400b63f0 con 0x7f90280043b0 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f903cff9640 1 -- 172.21.15.37:0/3721710516 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903000fd40 con 0x7f90280043b0 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.518+0000 7f9046924640 1 -- 172.21.15.37:0/3721710516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90400b69b0 con 0x7f90280043b0 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.519+0000 7f9046924640 1 -- 172.21.15.37:0/3721710516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904006a360 con 0x7f90280043b0 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.520+0000 7f903cff9640 1 -- 172.21.15.37:0/3721710516 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f903001d460 con 0x7f90280043b0 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.520+0000 7f903cff9640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f902c06c920 0x7f902c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.521+0000 7f903cff9640 1 -- 172.21.15.37:0/3721710516 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f903008bbf0 con 0x7f90280043b0 2024-04-26T20:22:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.521+0000 7f903f7fe640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f902c06c920 0x7f902c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.521+0000 7f903f7fe640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f902c06c920 0x7f902c06ede0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9034034000 tx=0x7f9034009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:43.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.525+0000 7f903cff9640 1 -- 172.21.15.37:0/3721710516 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f903005a020 con 0x7f90280043b0 2024-04-26T20:22:43.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:43.748+0000 7f9046924640 1 -- 172.21.15.37:0/3721710516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f904006d520 con 0x7f90280043b0 2024-04-26T20:22:44.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.751+0000 7f903cff9640 1 -- 172.21.15.37:0/3721710516 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) ==== 118+0+0 (secure 0 0 0) 0x7f9030059920 con 0x7f90280043b0 2024-04-26T20:22:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 -- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f902c06c920 msgr2=0x7f902c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f902c06c920 0x7f902c06ede0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9034034000 tx=0x7f9034009030 comp rx=0 tx=0).stop 2024-04-26T20:22:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 -- 172.21.15.37:0/3721710516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 msgr2=0x7f90401f5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 0x7f90401f5ca0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9030007310 tx=0x7f90300077d0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 -- 172.21.15.37:0/3721710516 shutdown_connections 2024-04-26T20:22:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f902c06c920 0x7f902c06ede0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9040159520 0x7f90401609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90401590a0 0x7f90400cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 --2- 172.21.15.37:0/3721710516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90280043b0 0x7f90401f5ca0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 -- 172.21.15.37:0/3721710516 >> 172.21.15.37:0/3721710516 conn(0x7f90400d2ff0 msgr2=0x7f90400d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:44.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 -- 172.21.15.37:0/3721710516 shutdown_connections 2024-04-26T20:22:44.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.754+0000 7f9046924640 1 -- 172.21.15.37:0/3721710516 wait complete. 2024-04-26T20:22:44.769 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:22:44.769 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:22:44.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.877+0000 7f71ccf4e640 1 Processor -- start 2024-04-26T20:22:44.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.877+0000 7f71ccf4e640 1 -- start start 2024-04-26T20:22:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71ccf4e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f71c80d5a80 0x7f71c80b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71ccf4e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f71c80b4b20 0x7f71c81630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71ccf4e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c80b2760 con 0x7f71c815f400 2024-04-26T20:22:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71ccf4e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f71c80b43e0 con 0x7f71c80d5a80 2024-04-26T20:22:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71ccf4e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f71c80b3ed0 con 0x7f71c80b4b20 2024-04-26T20:22:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71c659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f71c815f400 0x7f71c80b37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33116/0 (socket says 172.21.15.37:33116) 2024-04-26T20:22:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71c659c640 1 -- 172.21.15.37:0/3449163218 learned_addr learned my addr 172.21.15.37:0/3449163218 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71c5d9b640 1 --2- 172.21.15.37:0/3449163218 >> v2:172.21.15.37:3300/0 conn(0x7f71c80d5a80 0x7f71c80b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:44.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71c5d9b640 1 -- 172.21.15.37:0/3449163218 >> v2:172.21.15.134:3300/0 conn(0x7f71c80b4b20 msgr2=0x7f71c81630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:44.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71c6d9d640 1 --2- 172.21.15.37:0/3449163218 >> v2:172.21.15.134:3300/0 conn(0x7f71c80b4b20 0x7f71c81630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:44.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71c5d9b640 1 --2- 172.21.15.37:0/3449163218 >> v2:172.21.15.134:3300/0 conn(0x7f71c80b4b20 0x7f71c81630b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71c5d9b640 1 -- 172.21.15.37:0/3449163218 >> v1:172.21.15.37:6789/0 conn(0x7f71c815f400 legacy=0x7f71c80b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:44.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71c5d9b640 1 -- 172.21.15.37:0/3449163218 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c80b2a00 con 0x7f71c80d5a80 2024-04-26T20:22:44.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.878+0000 7f71c559a640 1 -- 172.21.15.37:0/3449163218 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1442923523 0 0) 0x7f71c80b2760 con 0x7f71c815f400 2024-04-26T20:22:44.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.879+0000 7f71c5d9b640 1 --2- 172.21.15.37:0/3449163218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80d5a80 0x7f71c80b45e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f71c80d28a0 tx=0x7f71bc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872bf06e8ebe61a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:44.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.879+0000 7f71c559a640 1 -- 172.21.15.37:0/3449163218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc037070 con 0x7f71c80d5a80 2024-04-26T20:22:44.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.879+0000 7f71c559a640 1 -- 172.21.15.37:0/3449163218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71bc032440 con 0x7f71c80d5a80 2024-04-26T20:22:44.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.879+0000 7f71c559a640 1 -- 172.21.15.37:0/3449163218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc02ae60 con 0x7f71c80d5a80 2024-04-26T20:22:44.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.880+0000 7f71ccf4e640 1 -- 172.21.15.37:0/3449163218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80d5a80 msgr2=0x7f71c80b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:44.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.880+0000 7f71ccf4e640 1 --2- 172.21.15.37:0/3449163218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80d5a80 0x7f71c80b45e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f71c80d28a0 tx=0x7f71bc02a110 comp rx=0 tx=0).stop 2024-04-26T20:22:44.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.880+0000 7f71ccf4e640 1 -- 172.21.15.37:0/3449163218 shutdown_connections 2024-04-26T20:22:44.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.880+0000 7f71ccf4e640 1 --2- 172.21.15.37:0/3449163218 >> v2:172.21.15.134:3300/0 conn(0x7f71c80b4b20 0x7f71c81630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.880+0000 7f71ccf4e640 1 --2- 172.21.15.37:0/3449163218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80d5a80 0x7f71c80b45e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.880+0000 7f71ccf4e640 1 -- 172.21.15.37:0/3449163218 >> 172.21.15.37:0/3449163218 conn(0x7f71c809e4f0 msgr2=0x7f71c80d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:44.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.880+0000 7f71ccf4e640 1 -- 172.21.15.37:0/3449163218 shutdown_connections 2024-04-26T20:22:44.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.880+0000 7f71ccf4e640 1 -- 172.21.15.37:0/3449163218 wait complete. 2024-04-26T20:22:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.881+0000 7f71ccf4e640 1 Processor -- start 2024-04-26T20:22:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.881+0000 7f71ccf4e640 1 -- start start 2024-04-26T20:22:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71ccf4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c80b4b20 0x7f71c8162410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71ccf4e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80d5a80 0x7f71c815fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71ccf4e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c815f400 0x7f71c815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71ccf4e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71c80b32a0 con 0x7f71c80d5a80 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71ccf4e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71c80b2fc0 con 0x7f71c80b4b20 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71ccf4e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f71c80b4f30 con 0x7f71c815f400 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71c5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80d5a80 0x7f71c815fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71c5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80d5a80 0x7f71c815fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48436/0 (socket says 172.21.15.37:48436) 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71c5d9b640 1 -- 172.21.15.37:0/1665136672 learned_addr learned my addr 172.21.15.37:0/1665136672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71c659c640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c80b4b20 0x7f71c8162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71c5d9b640 1 -- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c815f400 msgr2=0x7f71c815db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71c5d9b640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c815f400 0x7f71c815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71c5d9b640 1 -- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c80b4b20 msgr2=0x7f71c8162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71c5d9b640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c80b4b20 0x7f71c8162410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.882+0000 7f71c5d9b640 1 -- 172.21.15.37:0/1665136672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c80cefd0 con 0x7f71c80d5a80 2024-04-26T20:22:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.883+0000 7f71c5d9b640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80d5a80 0x7f71c815fb60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000c00 tx=0x7f71bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.883+0000 7f71b6ffd640 1 -- 172.21.15.37:0/1665136672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc037040 con 0x7f71c80d5a80 2024-04-26T20:22:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.883+0000 7f71b6ffd640 1 -- 172.21.15.37:0/1665136672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71bc042070 con 0x7f71c80d5a80 2024-04-26T20:22:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.883+0000 7f71ccf4e640 1 -- 172.21.15.37:0/1665136672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c80cc880 con 0x7f71c80d5a80 2024-04-26T20:22:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.883+0000 7f71b6ffd640 1 -- 172.21.15.37:0/1665136672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc030a50 con 0x7f71c80d5a80 2024-04-26T20:22:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.883+0000 7f71ccf4e640 1 -- 172.21.15.37:0/1665136672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c80cf270 con 0x7f71c80d5a80 2024-04-26T20:22:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.884+0000 7f71ccf4e640 1 -- 172.21.15.37:0/1665136672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c806a360 con 0x7f71c80d5a80 2024-04-26T20:22:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.884+0000 7f71b6ffd640 1 -- 172.21.15.37:0/1665136672 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71bc030bf0 con 0x7f71c80d5a80 2024-04-26T20:22:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.885+0000 7f71b6ffd640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f71a806c920 0x7f71a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.885+0000 7f71b6ffd640 1 -- 172.21.15.37:0/1665136672 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f71bc0ac530 con 0x7f71c80d5a80 2024-04-26T20:22:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.885+0000 7f71c659c640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f71a806c920 0x7f71a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.886+0000 7f71c659c640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f71a806c920 0x7f71a806ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f71b8000f60 tx=0x7f71b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:44.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:44.889+0000 7f71b6ffd640 1 -- 172.21.15.37:0/1665136672 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71bc07a9f0 con 0x7f71c80d5a80 2024-04-26T20:22:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.117+0000 7f71ccf4e640 1 -- 172.21.15.37:0/1665136672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f71c80cf4b0 con 0x7f71c80d5a80 2024-04-26T20:22:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.760+0000 7f71b6ffd640 1 -- 172.21.15.37:0/1665136672 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7f71bc07a2f0 con 0x7f71c80d5a80 2024-04-26T20:22:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.762+0000 7f71ccf4e640 1 -- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f71a806c920 msgr2=0x7f71a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.762+0000 7f71ccf4e640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f71a806c920 0x7f71a806ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f71b8000f60 tx=0x7f71b8009040 comp rx=0 tx=0).stop 2024-04-26T20:22:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.762+0000 7f71ccf4e640 1 -- 172.21.15.37:0/1665136672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80d5a80 msgr2=0x7f71c815fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.762+0000 7f71ccf4e640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80d5a80 0x7f71c815fb60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000c00 tx=0x7f71bc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.763+0000 7f71ccf4e640 1 -- 172.21.15.37:0/1665136672 shutdown_connections 2024-04-26T20:22:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.763+0000 7f71ccf4e640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f71a806c920 0x7f71a806ede0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.763+0000 7f71ccf4e640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c815f400 0x7f71c815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.763+0000 7f71ccf4e640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80d5a80 0x7f71c815fb60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.763+0000 7f71ccf4e640 1 --2- 172.21.15.37:0/1665136672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c80b4b20 0x7f71c8162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.763+0000 7f71ccf4e640 1 -- 172.21.15.37:0/1665136672 >> 172.21.15.37:0/1665136672 conn(0x7f71c809e4f0 msgr2=0x7f71c806df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.763+0000 7f71ccf4e640 1 -- 172.21.15.37:0/1665136672 shutdown_connections 2024-04-26T20:22:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.763+0000 7f71ccf4e640 1 -- 172.21.15.37:0/1665136672 wait complete. 2024-04-26T20:22:45.777 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:22:45.777 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:22:45.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.884+0000 7f852259e640 1 Processor -- start 2024-04-26T20:22:45.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.884+0000 7f852259e640 1 -- start start 2024-04-26T20:22:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.885+0000 7f852259e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f851c0d1fd0 0x7f851c0d3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.885+0000 7f852259e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f851c0d3a90 0x7f851c0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.885+0000 7f852259e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f851c0cc040 con 0x7f851c162870 2024-04-26T20:22:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.885+0000 7f852259e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f851c162c50 con 0x7f851c0d3a90 2024-04-26T20:22:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.885+0000 7f852259e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f851c0d2520 con 0x7f851c0d1fd0 2024-04-26T20:22:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.885+0000 7f8521d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f851c0d3a90 0x7f851c0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.885+0000 7f8521d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f851c0d3a90 0x7f851c0ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35798/0 (socket says 172.21.15.37:35798) 2024-04-26T20:22:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.885+0000 7f8521d9d640 1 -- 172.21.15.37:0/1682789780 learned_addr learned my addr 172.21.15.37:0/1682789780 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.886+0000 7f8520d9b640 1 --2- 172.21.15.37:0/1682789780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f851c0d1fd0 0x7f851c0d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.886+0000 7f8521d9d640 1 -- 172.21.15.37:0/1682789780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f851c0d1fd0 msgr2=0x7f851c0d3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.886+0000 7f8521d9d640 1 --2- 172.21.15.37:0/1682789780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f851c0d1fd0 0x7f851c0d3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.886+0000 7f8521d9d640 1 -- 172.21.15.37:0/1682789780 >> v1:172.21.15.134:6789/0 conn(0x7f851c162870 legacy=0x7f851c0d2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.886+0000 7f8521d9d640 1 -- 172.21.15.37:0/1682789780 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c0d2800 con 0x7f851c0d3a90 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.886+0000 7f8521d9d640 1 --2- 172.21.15.37:0/1682789780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f851c0d3a90 0x7f851c0ccae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8518009e50 tx=0x7f851802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70879a93eb8b707 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.887+0000 7f8513fff640 1 -- 172.21.15.37:0/1682789780 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518036070 con 0x7f851c0d3a90 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.887+0000 7f8513fff640 1 -- 172.21.15.37:0/1682789780 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8518031600 con 0x7f851c0d3a90 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.887+0000 7f8513fff640 1 -- 172.21.15.37:0/1682789780 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85180319d0 con 0x7f851c0d3a90 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.887+0000 7f852259e640 1 -- 172.21.15.37:0/1682789780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f851c0d3a90 msgr2=0x7f851c0ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.887+0000 7f852259e640 1 --2- 172.21.15.37:0/1682789780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f851c0d3a90 0x7f851c0ccae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8518009e50 tx=0x7f851802a630 comp rx=0 tx=0).stop 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.887+0000 7f852259e640 1 -- 172.21.15.37:0/1682789780 shutdown_connections 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.887+0000 7f852259e640 1 --2- 172.21.15.37:0/1682789780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f851c0d3a90 0x7f851c0ccae0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.887+0000 7f852259e640 1 --2- 172.21.15.37:0/1682789780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f851c0d1fd0 0x7f851c0d3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.887+0000 7f852259e640 1 -- 172.21.15.37:0/1682789780 >> 172.21.15.37:0/1682789780 conn(0x7f851c0d4f70 msgr2=0x7f851c0d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.887+0000 7f852259e640 1 -- 172.21.15.37:0/1682789780 shutdown_connections 2024-04-26T20:22:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.887+0000 7f852259e640 1 -- 172.21.15.37:0/1682789780 wait complete. 2024-04-26T20:22:45.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.889+0000 7f852259e640 1 Processor -- start 2024-04-26T20:22:45.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.889+0000 7f852259e640 1 -- start start 2024-04-26T20:22:45.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.889+0000 7f852259e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f851c0d1fd0 0x7f851c15efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:45.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.889+0000 7f852259e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f851c0d3a90 0x7f851c0ba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:45.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.889+0000 7f852259e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f851c162870 0x7f851c0ba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:45.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.889+0000 7f852259e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0cd020 con 0x7f851c0d3a90 2024-04-26T20:22:45.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.889+0000 7f852259e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0cc5c0 con 0x7f851c0d1fd0 2024-04-26T20:22:45.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.889+0000 7f852259e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f851c0cd1a0 con 0x7f851c162870 2024-04-26T20:22:45.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.889+0000 7f8520d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f851c0d3a90 0x7f851c0ba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:45.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8520d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f851c0d3a90 0x7f851c0ba0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35800/0 (socket says 172.21.15.37:35800) 2024-04-26T20:22:45.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8520d9b640 1 -- 172.21.15.37:0/1270774612 learned_addr learned my addr 172.21.15.37:0/1270774612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:45.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8521d9d640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f851c162870 0x7f851c0ba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:45.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f852159c640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f851c0d1fd0 0x7f851c15efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:45.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8520d9b640 1 -- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f851c162870 msgr2=0x7f851c0ba5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:45.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8520d9b640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f851c162870 0x7f851c0ba5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8520d9b640 1 -- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f851c0d1fd0 msgr2=0x7f851c15efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:45.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8520d9b640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f851c0d1fd0 0x7f851c15efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8520d9b640 1 -- 172.21.15.37:0/1270774612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c15f4f0 con 0x7f851c0d3a90 2024-04-26T20:22:45.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8520d9b640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f851c0d3a90 0x7f851c0ba0b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f850c00d800 tx=0x7f850c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:45.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8511ffb640 1 -- 172.21.15.37:0/1270774612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c00b6c0 con 0x7f851c0d3a90 2024-04-26T20:22:45.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8511ffb640 1 -- 172.21.15.37:0/1270774612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850c019070 con 0x7f851c0d3a90 2024-04-26T20:22:45.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f852259e640 1 -- 172.21.15.37:0/1270774612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c0b1c70 con 0x7f851c0d3a90 2024-04-26T20:22:45.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f8511ffb640 1 -- 172.21.15.37:0/1270774612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c00ba80 con 0x7f851c0d3a90 2024-04-26T20:22:45.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.890+0000 7f852259e640 1 -- 172.21.15.37:0/1270774612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c0b3b90 con 0x7f851c0d3a90 2024-04-26T20:22:45.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.891+0000 7f852259e640 1 -- 172.21.15.37:0/1270774612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851c076890 con 0x7f851c0d3a90 2024-04-26T20:22:45.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.892+0000 7f8511ffb640 1 -- 172.21.15.37:0/1270774612 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f850c00bc20 con 0x7f851c0d3a90 2024-04-26T20:22:45.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.892+0000 7f8511ffb640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f850006c8f0 0x7f850006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:45.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.893+0000 7f8511ffb640 1 -- 172.21.15.37:0/1270774612 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f850c08bf70 con 0x7f851c0d3a90 2024-04-26T20:22:45.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.893+0000 7f852159c640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f850006c8f0 0x7f850006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:45.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.893+0000 7f852159c640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f850006c8f0 0x7f850006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8514006fd0 tx=0x7f8514008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:45.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:45.897+0000 7f8511ffb640 1 -- 172.21.15.37:0/1270774612 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f851c15f4f0 con 0x7f851c0d3a90 2024-04-26T20:22:46.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.127+0000 7f852259e640 1 -- 172.21.15.37:0/1270774612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f851c074760 con 0x7f851c0d3a90 2024-04-26T20:22:46.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.127+0000 7f8511ffb640 1 -- 172.21.15.37:0/1270774612 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5181 (secure 0 0 0) 0x7f850c011340 con 0x7f851c0d3a90 2024-04-26T20:22:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:46.128 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/4166977617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4166977617},{"type":"v1","addr":"172.21.15.134:6836","nonce":4166977617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3245561282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3245561282},{"type":"v1","addr":"172.21.15.134:6837","nonce":3245561282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3853963851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3853963851},{"type":"v1","addr":"172.21.15.37:6835","nonce":3853963851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:22:42.116910+0000","modified":"2024-04-26T20:22:45.119356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5097":{"gid":5097,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/64661056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":64661056},{"type":"v1","addr":"172.21.15.37:6834","nonce":64661056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5097,"qdb_cluster":[5097]},"id":2}]} 2024-04-26T20:22:46.129 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 15 2024-04-26T20:22:46.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 -- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f850006c8f0 msgr2=0x7f850006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f850006c8f0 0x7f850006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8514006fd0 tx=0x7f8514008040 comp rx=0 tx=0).stop 2024-04-26T20:22:46.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 -- 172.21.15.37:0/1270774612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f851c0d3a90 msgr2=0x7f851c0ba0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f851c0d3a90 0x7f851c0ba0b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f850c00d800 tx=0x7f850c00dcc0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 -- 172.21.15.37:0/1270774612 shutdown_connections 2024-04-26T20:22:46.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f850006c8f0 0x7f850006edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f851c162870 0x7f851c0ba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f851c0d3a90 0x7f851c0ba0b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 --2- 172.21.15.37:0/1270774612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f851c0d1fd0 0x7f851c15efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 -- 172.21.15.37:0/1270774612 >> 172.21.15.37:0/1270774612 conn(0x7f851c0d4f70 msgr2=0x7f851c1619b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:46.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 -- 172.21.15.37:0/1270774612 shutdown_connections 2024-04-26T20:22:46.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.129+0000 7f852259e640 1 -- 172.21.15.37:0/1270774612 wait complete. 2024-04-26T20:22:46.141 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:22:46.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.240+0000 7f0a48c12640 1 Processor -- start 2024-04-26T20:22:46.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.240+0000 7f0a48c12640 1 -- start start 2024-04-26T20:22:46.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.241+0000 7f0a48c12640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4406ea40 0x7f0a440d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.241+0000 7f0a48c12640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0a440cf830 0x7f0a440cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.241+0000 7f0a48c12640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a440b1d70 con 0x7f0a44162e40 2024-04-26T20:22:46.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.241+0000 7f0a48c12640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f0a440d01e0 con 0x7f0a440cf830 2024-04-26T20:22:46.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.241+0000 7f0a48c12640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0a44163220 con 0x7f0a4406ea40 2024-04-26T20:22:46.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.242+0000 7f0a41d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4406ea40 0x7f0a440d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.242+0000 7f0a42d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0a440cf830 0x7f0a440cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.242+0000 7f0a41d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4406ea40 0x7f0a440d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:50428/0 (socket says 172.21.15.37:50428) 2024-04-26T20:22:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.242+0000 7f0a41d9b640 1 -- 172.21.15.37:0/2851166284 learned_addr learned my addr 172.21.15.37:0/2851166284 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.242+0000 7f0a41d9b640 1 -- 172.21.15.37:0/2851166284 >> v2:172.21.15.134:3300/0 conn(0x7f0a440cf830 msgr2=0x7f0a440cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.242+0000 7f0a41d9b640 1 --2- 172.21.15.37:0/2851166284 >> v2:172.21.15.134:3300/0 conn(0x7f0a440cf830 0x7f0a440cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.242+0000 7f0a41d9b640 1 -- 172.21.15.37:0/2851166284 >> v1:172.21.15.134:6789/0 conn(0x7f0a44162e40 legacy=0x7f0a440b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.242+0000 7f0a41d9b640 1 -- 172.21.15.37:0/2851166284 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a440d1960 con 0x7f0a4406ea40 2024-04-26T20:22:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.243+0000 7f0a41d9b640 1 --2- 172.21.15.37:0/2851166284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4406ea40 0x7f0a440d1340 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009870 tx=0x7f0a3802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11edea01ed84fe9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.243+0000 7f0a4159a640 1 -- 172.21.15.37:0/2851166284 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38037070 con 0x7f0a4406ea40 2024-04-26T20:22:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.243+0000 7f0a4159a640 1 -- 172.21.15.37:0/2851166284 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a38032440 con 0x7f0a4406ea40 2024-04-26T20:22:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.243+0000 7f0a4159a640 1 -- 172.21.15.37:0/2851166284 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38032810 con 0x7f0a4406ea40 2024-04-26T20:22:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.244+0000 7f0a48c12640 1 -- 172.21.15.37:0/2851166284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4406ea40 msgr2=0x7f0a440d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.244+0000 7f0a48c12640 1 --2- 172.21.15.37:0/2851166284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4406ea40 0x7f0a440d1340 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009870 tx=0x7f0a3802a1c0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.244+0000 7f0a48c12640 1 -- 172.21.15.37:0/2851166284 shutdown_connections 2024-04-26T20:22:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.244+0000 7f0a48c12640 1 --2- 172.21.15.37:0/2851166284 >> v2:172.21.15.134:3300/0 conn(0x7f0a440cf830 0x7f0a440cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.244+0000 7f0a48c12640 1 --2- 172.21.15.37:0/2851166284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4406ea40 0x7f0a440d1340 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.244+0000 7f0a48c12640 1 -- 172.21.15.37:0/2851166284 >> 172.21.15.37:0/2851166284 conn(0x7f0a4409ba50 msgr2=0x7f0a4409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.244+0000 7f0a48c12640 1 -- 172.21.15.37:0/2851166284 shutdown_connections 2024-04-26T20:22:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.244+0000 7f0a48c12640 1 -- 172.21.15.37:0/2851166284 wait complete. 2024-04-26T20:22:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.245+0000 7f0a48c12640 1 Processor -- start 2024-04-26T20:22:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.245+0000 7f0a48c12640 1 -- start start 2024-04-26T20:22:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a48c12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a4406ea40 0x7f0a441f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a48c12640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a440cf830 0x7f0a440c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a48c12640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a44162e40 0x7f0a440b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a48c12640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0a440d1f20 con 0x7f0a4406ea40 2024-04-26T20:22:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a48c12640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a440d0eb0 con 0x7f0a44162e40 2024-04-26T20:22:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a48c12640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0a440ce010 con 0x7f0a440cf830 2024-04-26T20:22:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a4259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a4406ea40 0x7f0a441f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a4259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a4406ea40 0x7f0a441f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35804/0 (socket says 172.21.15.37:35804) 2024-04-26T20:22:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a4259c640 1 -- 172.21.15.37:0/160977739 learned_addr learned my addr 172.21.15.37:0/160977739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a41d9b640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a440cf830 0x7f0a440c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a42d9d640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a44162e40 0x7f0a440b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a4259c640 1 -- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a440cf830 msgr2=0x7f0a440c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a4259c640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a440cf830 0x7f0a440c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a4259c640 1 -- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a44162e40 msgr2=0x7f0a440b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a4259c640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a44162e40 0x7f0a440b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.246+0000 7f0a4259c640 1 -- 172.21.15.37:0/160977739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a440ca120 con 0x7f0a4406ea40 2024-04-26T20:22:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.247+0000 7f0a4259c640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a4406ea40 0x7f0a441f4820 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c3c0 tx=0x7f0a3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.247+0000 7f0a32ffd640 1 -- 172.21.15.37:0/160977739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a340052d0 con 0x7f0a4406ea40 2024-04-26T20:22:46.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.247+0000 7f0a32ffd640 1 -- 172.21.15.37:0/160977739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a34005490 con 0x7f0a4406ea40 2024-04-26T20:22:46.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.247+0000 7f0a48c12640 1 -- 172.21.15.37:0/160977739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a440ca6e0 con 0x7f0a4406ea40 2024-04-26T20:22:46.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.247+0000 7f0a32ffd640 1 -- 172.21.15.37:0/160977739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34007260 con 0x7f0a4406ea40 2024-04-26T20:22:46.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.247+0000 7f0a48c12640 1 -- 172.21.15.37:0/160977739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a440caca0 con 0x7f0a4406ea40 2024-04-26T20:22:46.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.248+0000 7f0a48c12640 1 -- 172.21.15.37:0/160977739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a4406a360 con 0x7f0a4406ea40 2024-04-26T20:22:46.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.248+0000 7f0a32ffd640 1 -- 172.21.15.37:0/160977739 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a34007400 con 0x7f0a4406ea40 2024-04-26T20:22:46.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.249+0000 7f0a32ffd640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0a2406c8f0 0x7f0a2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.249+0000 7f0a32ffd640 1 -- 172.21.15.37:0/160977739 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0a3408b040 con 0x7f0a4406ea40 2024-04-26T20:22:46.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.249+0000 7f0a41d9b640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0a2406c8f0 0x7f0a2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.249+0000 7f0a41d9b640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0a2406c8f0 0x7f0a2406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a38034000 tx=0x7f0a3802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:46.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.253+0000 7f0a32ffd640 1 -- 172.21.15.37:0/160977739 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a34058d80 con 0x7f0a4406ea40 2024-04-26T20:22:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.437+0000 7f0a48c12640 1 -- 172.21.15.37:0/160977739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a440cfe20 con 0x7f0a4406ea40 2024-04-26T20:22:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.438+0000 7f0a32ffd640 1 -- 172.21.15.37:0/160977739 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16681 (secure 0 0 0) 0x7f0a3405d080 con 0x7f0a4406ea40 2024-04-26T20:22:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:46.440 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:22:43.746058+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:22:39.762703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:22:41.066360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:22:46.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.440+0000 7f0a48c12640 1 -- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0a2406c8f0 msgr2=0x7f0a2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.440+0000 7f0a48c12640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0a2406c8f0 0x7f0a2406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a38034000 tx=0x7f0a3802e000 comp rx=0 tx=0).stop 2024-04-26T20:22:46.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.440+0000 7f0a48c12640 1 -- 172.21.15.37:0/160977739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a4406ea40 msgr2=0x7f0a441f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.440+0000 7f0a48c12640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a4406ea40 0x7f0a441f4820 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c3c0 tx=0x7f0a3400c880 comp rx=0 tx=0).stop 2024-04-26T20:22:46.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.440+0000 7f0a48c12640 1 -- 172.21.15.37:0/160977739 shutdown_connections 2024-04-26T20:22:46.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.440+0000 7f0a48c12640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a44162e40 0x7f0a440b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.441+0000 7f0a48c12640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0a2406c8f0 0x7f0a2406edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.441+0000 7f0a48c12640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a440cf830 0x7f0a440c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.441+0000 7f0a48c12640 1 --2- 172.21.15.37:0/160977739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a4406ea40 0x7f0a441f4820 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.441+0000 7f0a48c12640 1 -- 172.21.15.37:0/160977739 >> 172.21.15.37:0/160977739 conn(0x7f0a4409ba50 msgr2=0x7f0a440cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:46.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.441+0000 7f0a48c12640 1 -- 172.21.15.37:0/160977739 shutdown_connections 2024-04-26T20:22:46.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.441+0000 7f0a48c12640 1 -- 172.21.15.37:0/160977739 wait complete. 2024-04-26T20:22:46.453 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:22:46.453 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:22:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.553+0000 7f1351919640 1 Processor -- start 2024-04-26T20:22:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.553+0000 7f1351919640 1 -- start start 2024-04-26T20:22:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f1351919640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f134c159a90 0x7f134c06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f1351919640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134c072e30 con 0x7f134c160320 2024-04-26T20:22:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f1351919640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134c073110 con 0x7f134c070130 2024-04-26T20:22:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f1351919640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f134c0733f0 con 0x7f134c159a90 2024-04-26T20:22:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f134c160320 0x7f134c0ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52566/0 (socket says 172.21.15.37:52566) 2024-04-26T20:22:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134b7fe640 1 -- 172.21.15.37:0/452803603 learned_addr learned my addr 172.21.15.37:0/452803603 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134affd640 1 --2- 172.21.15.37:0/452803603 >> v2:172.21.15.134:3300/0 conn(0x7f134c159a90 0x7f134c06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2336907512 0 0) 0x7f134c072e30 con 0x7f134c160320 2024-04-26T20:22:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1338003520 con 0x7f134c160320 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1984442836 0 0) 0x7f1338003520 con 0x7f134c160320 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 >> v2:172.21.15.134:3300/0 conn(0x7f134c159a90 msgr2=0x7f134c06fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134a7fc640 1 --2- 172.21.15.37:0/452803603 >> v2:172.21.15.134:3300/0 conn(0x7f134c159a90 0x7f134c06fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 >> v1:172.21.15.134:6789/0 conn(0x7f134c070130 legacy=0x7f134c15fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c0736d0 con 0x7f134c160320 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f13440019f0 con 0x7f134c160320 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.554+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 >> v1:172.21.15.37:6789/0 conn(0x7f134c160320 legacy=0x7f134c0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f134a7fc640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13380039b0 0x7f1338003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f134a7fc640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1338004800 0x7f1338004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f134a7fc640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13380051c0 0x7f133800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f134c0733f0 con 0x7f1338004800 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f133800c110 con 0x7f13380051c0 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f133800c310 con 0x7f13380039b0 2024-04-26T20:22:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f1343fff640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1338004800 0x7f1338004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f134affd640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13380039b0 0x7f1338003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f134b7fe640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13380051c0 0x7f133800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f1343fff640 1 -- 172.21.15.37:0/452803603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13380039b0 msgr2=0x7f1338003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f1343fff640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13380039b0 0x7f1338003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f1343fff640 1 -- 172.21.15.37:0/452803603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13380051c0 msgr2=0x7f133800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f1343fff640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13380051c0 0x7f133800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f1343fff640 1 -- 172.21.15.37:0/452803603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1338003520 con 0x7f1338004800 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f134b7fe640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13380051c0 0x7f133800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.555+0000 7f134b7fe640 1 -- 172.21.15.37:0/452803603 reap_dead start 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.556+0000 7f1343fff640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1338004800 0x7f1338004c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f134c070580 tx=0x7f1338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48aab71ddd0e5829 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.556+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334037070 con 0x7f1338004800 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.556+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133402ab90 con 0x7f1338004800 2024-04-26T20:22:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.556+0000 7f134a7fc640 1 -- 172.21.15.37:0/452803603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334001430 con 0x7f1338004800 2024-04-26T20:22:46.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.556+0000 7f1351919640 1 -- 172.21.15.37:0/452803603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1338004800 msgr2=0x7f1338004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.556+0000 7f1351919640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1338004800 0x7f1338004c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f134c070580 tx=0x7f1338005830 comp rx=0 tx=0).stop 2024-04-26T20:22:46.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.556+0000 7f1351919640 1 -- 172.21.15.37:0/452803603 shutdown_connections 2024-04-26T20:22:46.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.556+0000 7f1351919640 1 --2- 172.21.15.37:0/452803603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1338004800 0x7f1338004c80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.556+0000 7f1351919640 1 -- 172.21.15.37:0/452803603 >> 172.21.15.37:0/452803603 conn(0x7f134c0d0d50 msgr2=0x7f134c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:46.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.556+0000 7f1351919640 1 -- 172.21.15.37:0/452803603 shutdown_connections 2024-04-26T20:22:46.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.557+0000 7f1351919640 1 -- 172.21.15.37:0/452803603 wait complete. 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.558+0000 7f1351919640 1 Processor -- start 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.558+0000 7f1351919640 1 -- start start 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.558+0000 7f1351919640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1338004800 0x7f134c1f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.558+0000 7f1351919640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f134c152110 0x7f134c0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.558+0000 7f1351919640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c153720 0x7f134c15e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.558+0000 7f1351919640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f134c159ff0 con 0x7f134c153720 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.558+0000 7f1351919640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f134c159e70 con 0x7f134c152110 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.558+0000 7f1351919640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f134c15fc10 con 0x7f1338004800 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f134b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c153720 0x7f134c15e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f134b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c153720 0x7f134c15e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35828/0 (socket says 172.21.15.37:35828) 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f134b7fe640 1 -- 172.21.15.37:0/3393628884 learned_addr learned my addr 172.21.15.37:0/3393628884 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f134affd640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1338004800 0x7f134c1f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f1343fff640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f134c152110 0x7f134c0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f134b7fe640 1 -- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1338004800 msgr2=0x7f134c1f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f134b7fe640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1338004800 0x7f134c1f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f134b7fe640 1 -- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f134c152110 msgr2=0x7f134c0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f134b7fe640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f134c152110 0x7f134c0cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f134b7fe640 1 -- 172.21.15.37:0/3393628884 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c152680 con 0x7f134c153720 2024-04-26T20:22:46.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f134b7fe640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c153720 0x7f134c15e6b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f134c0b3b40 tx=0x7f1344007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:46.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f1348ff9640 1 -- 172.21.15.37:0/3393628884 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134400e960 con 0x7f134c153720 2024-04-26T20:22:46.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f1348ff9640 1 -- 172.21.15.37:0/3393628884 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134400eb00 con 0x7f134c153720 2024-04-26T20:22:46.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f1351919640 1 -- 172.21.15.37:0/3393628884 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134c152960 con 0x7f134c153720 2024-04-26T20:22:46.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f1348ff9640 1 -- 172.21.15.37:0/3393628884 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344014480 con 0x7f134c153720 2024-04-26T20:22:46.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.559+0000 7f1351919640 1 -- 172.21.15.37:0/3393628884 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134c152f20 con 0x7f134c153720 2024-04-26T20:22:46.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.560+0000 7f1351919640 1 -- 172.21.15.37:0/3393628884 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134c06a360 con 0x7f134c153720 2024-04-26T20:22:46.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.561+0000 7f1348ff9640 1 -- 172.21.15.37:0/3393628884 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1344014620 con 0x7f134c153720 2024-04-26T20:22:46.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.561+0000 7f1348ff9640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f132c06c8f0 0x7f132c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.561+0000 7f1348ff9640 1 -- 172.21.15.37:0/3393628884 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f134408b890 con 0x7f134c153720 2024-04-26T20:22:46.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.561+0000 7f134affd640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f132c06c8f0 0x7f132c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.562+0000 7f134affd640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f132c06c8f0 0x7f132c06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f133c009c40 tx=0x7f133c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:46.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.565+0000 7f1348ff9640 1 -- 172.21.15.37:0/3393628884 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1344059cc0 con 0x7f134c153720 2024-04-26T20:22:46.716 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-26T20:22:46.715+0000 7fb78cee4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:22:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.747+0000 7f1351919640 1 -- 172.21.15.37:0/3393628884 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f134c06f980 con 0x7f132c06c8f0 2024-04-26T20:22:46.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.754+0000 7f1348ff9640 1 -- 172.21.15.37:0/3393628884 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7f134c06f980 con 0x7f132c06c8f0 2024-04-26T20:22:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:46.760 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:22:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 -- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f132c06c8f0 msgr2=0x7f132c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f132c06c8f0 0x7f132c06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f133c009c40 tx=0x7f133c009190 comp rx=0 tx=0).stop 2024-04-26T20:22:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 -- 172.21.15.37:0/3393628884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c153720 msgr2=0x7f134c15e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c153720 0x7f134c15e6b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f134c0b3b40 tx=0x7f1344007b00 comp rx=0 tx=0).stop 2024-04-26T20:22:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 -- 172.21.15.37:0/3393628884 shutdown_connections 2024-04-26T20:22:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c153720 0x7f134c15e6b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f134c152110 0x7f134c0cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f132c06c8f0 0x7f132c06edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 --2- 172.21.15.37:0/3393628884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1338004800 0x7f134c1f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 -- 172.21.15.37:0/3393628884 >> 172.21.15.37:0/3393628884 conn(0x7f134c0d0d50 msgr2=0x7f134c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:46.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 -- 172.21.15.37:0/3393628884 shutdown_connections 2024-04-26T20:22:46.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.757+0000 7f1351919640 1 -- 172.21.15.37:0/3393628884 wait complete. 2024-04-26T20:22:46.769 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-04-26T20:22:45.346702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220008,"kb_used_data":3832,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754712,"statfs":{"total":773094113280,"available":772868825088,"internally_reserved":0,"allocated":3923968,"data_stored":2223996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.613063"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.747315+0000","last_change":"2024-04-26T20:22:41.747315+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.747315+0000","last_undegraded":"2024-04-26T20:22:41.747315+0000","last_fullsized":"2024-04-26T20:22:41.747315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751176+0000","last_change":"2024-04-26T20:22:41.751176+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751176+0000","last_undegraded":"2024-04-26T20:22:41.751176+0000","last_fullsized":"2024-04-26T20:22:41.751176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749201+0000","last_change":"2024-04-26T20:22:41.749201+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749201+0000","last_undegraded":"2024-04-26T20:22:41.749201+0000","last_fullsized":"2024-04-26T20:22:41.749201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745545+0000","last_change":"2024-04-26T20:22:41.745545+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745545+0000","last_undegraded":"2024-04-26T20:22:41.745545+0000","last_fullsized":"2024-04-26T20:22:41.745545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745404+0000","last_change":"2024-04-26T20:22:41.745404+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745404+0000","last_undegraded":"2024-04-26T20:22:41.745404+0000","last_fullsized":"2024-04-26T20:22:41.745404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750104+0000","last_change":"2024-04-26T20:22:41.750104+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750104+0000","last_undegraded":"2024-04-26T20:22:41.750104+0000","last_fullsized":"2024-04-26T20:22:41.750104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750969+0000","last_change":"2024-04-26T20:22:41.750969+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750969+0000","last_undegraded":"2024-04-26T20:22:41.750969+0000","last_fullsized":"2024-04-26T20:22:41.750969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753949+0000","last_change":"2024-04-26T20:22:41.753949+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753949+0000","last_undegraded":"2024-04-26T20:22:41.753949+0000","last_fullsized":"2024-04-26T20:22:41.753949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749215+0000","last_change":"2024-04-26T20:22:41.749215+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749215+0000","last_undegraded":"2024-04-26T20:22:41.749215+0000","last_fullsized":"2024-04-26T20:22:41.749215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744734+0000","last_change":"2024-04-26T20:22:41.744734+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744734+0000","last_undegraded":"2024-04-26T20:22:41.744734+0000","last_fullsized":"2024-04-26T20:22:41.744734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751008+0000","last_change":"2024-04-26T20:22:41.751008+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751008+0000","last_undegraded":"2024-04-26T20:22:41.751008+0000","last_fullsized":"2024-04-26T20:22:41.751008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751513+0000","last_change":"2024-04-26T20:22:41.751513+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751513+0000","last_undegraded":"2024-04-26T20:22:41.751513+0000","last_fullsized":"2024-04-26T20:22:41.751513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744914+0000","last_change":"2024-04-26T20:22:41.744914+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744914+0000","last_undegraded":"2024-04-26T20:22:41.744914+0000","last_fullsized":"2024-04-26T20:22:41.744914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749030+0000","last_change":"2024-04-26T20:22:41.749030+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749030+0000","last_undegraded":"2024-04-26T20:22:41.749030+0000","last_fullsized":"2024-04-26T20:22:41.749030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750437+0000","last_change":"2024-04-26T20:22:41.750437+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750437+0000","last_undegraded":"2024-04-26T20:22:41.750437+0000","last_fullsized":"2024-04-26T20:22:41.750437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751270+0000","last_change":"2024-04-26T20:22:41.751270+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751270+0000","last_undegraded":"2024-04-26T20:22:41.751270+0000","last_fullsized":"2024-04-26T20:22:41.751270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751848+0000","last_change":"2024-04-26T20:22:41.751848+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751848+0000","last_undegraded":"2024-04-26T20:22:41.751848+0000","last_fullsized":"2024-04-26T20:22:41.751848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750736+0000","last_change":"2024-04-26T20:22:41.750736+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750736+0000","last_undegraded":"2024-04-26T20:22:41.750736+0000","last_fullsized":"2024-04-26T20:22:41.750736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750525+0000","last_change":"2024-04-26T20:22:41.750525+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750525+0000","last_undegraded":"2024-04-26T20:22:41.750525+0000","last_fullsized":"2024-04-26T20:22:41.750525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744169+0000","last_change":"2024-04-26T20:22:41.744169+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744169+0000","last_undegraded":"2024-04-26T20:22:41.744169+0000","last_fullsized":"2024-04-26T20:22:41.744169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748990+0000","last_change":"2024-04-26T20:22:41.748990+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748990+0000","last_undegraded":"2024-04-26T20:22:41.748990+0000","last_fullsized":"2024-04-26T20:22:41.748990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750071+0000","last_change":"2024-04-26T20:22:41.750071+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750071+0000","last_undegraded":"2024-04-26T20:22:41.750071+0000","last_fullsized":"2024-04-26T20:22:41.750071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744336+0000","last_change":"2024-04-26T20:22:41.744336+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744336+0000","last_undegraded":"2024-04-26T20:22:41.744336+0000","last_fullsized":"2024-04-26T20:22:41.744336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749073+0000","last_change":"2024-04-26T20:22:41.749073+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749073+0000","last_undegraded":"2024-04-26T20:22:41.749073+0000","last_fullsized":"2024-04-26T20:22:41.749073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745963+0000","last_change":"2024-04-26T20:22:41.745963+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745963+0000","last_undegraded":"2024-04-26T20:22:41.745963+0000","last_fullsized":"2024-04-26T20:22:41.745963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750267+0000","last_change":"2024-04-26T20:22:41.750267+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750267+0000","last_undegraded":"2024-04-26T20:22:41.750267+0000","last_fullsized":"2024-04-26T20:22:41.750267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744351+0000","last_change":"2024-04-26T20:22:41.744351+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744351+0000","last_undegraded":"2024-04-26T20:22:41.744351+0000","last_fullsized":"2024-04-26T20:22:41.744351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748823+0000","last_change":"2024-04-26T20:22:41.748823+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748823+0000","last_undegraded":"2024-04-26T20:22:41.748823+0000","last_fullsized":"2024-04-26T20:22:41.748823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748525+0000","last_change":"2024-04-26T20:22:41.748525+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748525+0000","last_undegraded":"2024-04-26T20:22:41.748525+0000","last_fullsized":"2024-04-26T20:22:41.748525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.752597+0000","last_change":"2024-04-26T20:22:41.752597+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.752597+0000","last_undegraded":"2024-04-26T20:22:41.752597+0000","last_fullsized":"2024-04-26T20:22:41.752597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.746522+0000","last_change":"2024-04-26T20:22:41.746522+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.746522+0000","last_undegraded":"2024-04-26T20:22:41.746522+0000","last_fullsized":"2024-04-26T20:22:41.746522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753519+0000","last_change":"2024-04-26T20:22:41.753519+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753519+0000","last_undegraded":"2024-04-26T20:22:41.753519+0000","last_fullsized":"2024-04-26T20:22:41.753519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744224+0000","last_change":"2024-04-26T20:22:41.744224+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744224+0000","last_undegraded":"2024-04-26T20:22:41.744224+0000","last_fullsized":"2024-04-26T20:22:41.744224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760557+0000","last_change":"2024-04-26T20:22:41.756667+0000","last_active":"2024-04-26T20:22:41.760557+0000","last_peered":"2024-04-26T20:22:41.760557+0000","last_clean":"2024-04-26T20:22:41.760557+0000","last_became_active":"2024-04-26T20:22:41.756385+0000","last_became_peered":"2024-04-26T20:22:41.756385+0000","last_unstale":"2024-04-26T20:22:41.760557+0000","last_undegraded":"2024-04-26T20:22:41.760557+0000","last_fullsized":"2024-04-26T20:22:41.760557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.746845+0000","last_change":"2024-04-26T20:22:41.746845+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.746845+0000","last_undegraded":"2024-04-26T20:22:41.746845+0000","last_fullsized":"2024-04-26T20:22:41.746845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764642+0000","last_change":"2024-04-26T20:22:41.758415+0000","last_active":"2024-04-26T20:22:41.764642+0000","last_peered":"2024-04-26T20:22:41.764642+0000","last_clean":"2024-04-26T20:22:41.764642+0000","last_became_active":"2024-04-26T20:22:41.757712+0000","last_became_peered":"2024-04-26T20:22:41.757712+0000","last_unstale":"2024-04-26T20:22:41.764642+0000","last_undegraded":"2024-04-26T20:22:41.764642+0000","last_fullsized":"2024-04-26T20:22:41.764642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:38.972810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749597+0000","last_change":"2024-04-26T20:22:41.749597+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749597+0000","last_undegraded":"2024-04-26T20:22:41.749597+0000","last_fullsized":"2024-04-26T20:22:41.749597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764752+0000","last_change":"2024-04-26T20:22:41.759486+0000","last_active":"2024-04-26T20:22:41.764752+0000","last_peered":"2024-04-26T20:22:41.764752+0000","last_clean":"2024-04-26T20:22:41.764752+0000","last_became_active":"2024-04-26T20:22:41.759076+0000","last_became_peered":"2024-04-26T20:22:41.759076+0000","last_unstale":"2024-04-26T20:22:41.764752+0000","last_undegraded":"2024-04-26T20:22:41.764752+0000","last_fullsized":"2024-04-26T20:22:41.764752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:27.337131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743760+0000","last_change":"2024-04-26T20:22:41.743760+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743760+0000","last_undegraded":"2024-04-26T20:22:41.743760+0000","last_fullsized":"2024-04-26T20:22:41.743760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758241+0000","last_change":"2024-04-26T20:22:41.754766+0000","last_active":"2024-04-26T20:22:41.758241+0000","last_peered":"2024-04-26T20:22:41.758241+0000","last_clean":"2024-04-26T20:22:41.758241+0000","last_became_active":"2024-04-26T20:22:41.754427+0000","last_became_peered":"2024-04-26T20:22:41.754427+0000","last_unstale":"2024-04-26T20:22:41.758241+0000","last_undegraded":"2024-04-26T20:22:41.758241+0000","last_fullsized":"2024-04-26T20:22:41.758241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:52.253630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750686+0000","last_change":"2024-04-26T20:22:41.750686+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750686+0000","last_undegraded":"2024-04-26T20:22:41.750686+0000","last_fullsized":"2024-04-26T20:22:41.750686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762211+0000","last_change":"2024-04-26T20:22:41.759951+0000","last_active":"2024-04-26T20:22:41.762211+0000","last_peered":"2024-04-26T20:22:41.762211+0000","last_clean":"2024-04-26T20:22:41.762211+0000","last_became_active":"2024-04-26T20:22:41.759681+0000","last_became_peered":"2024-04-26T20:22:41.759681+0000","last_unstale":"2024-04-26T20:22:41.762211+0000","last_undegraded":"2024-04-26T20:22:41.762211+0000","last_fullsized":"2024-04-26T20:22:41.762211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:49.082266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750653+0000","last_change":"2024-04-26T20:22:41.750653+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750653+0000","last_undegraded":"2024-04-26T20:22:41.750653+0000","last_fullsized":"2024-04-26T20:22:41.750653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761855+0000","last_change":"2024-04-26T20:22:41.758461+0000","last_active":"2024-04-26T20:22:41.761855+0000","last_peered":"2024-04-26T20:22:41.761855+0000","last_clean":"2024-04-26T20:22:41.761855+0000","last_became_active":"2024-04-26T20:22:41.758166+0000","last_became_peered":"2024-04-26T20:22:41.758166+0000","last_unstale":"2024-04-26T20:22:41.761855+0000","last_undegraded":"2024-04-26T20:22:41.761855+0000","last_fullsized":"2024-04-26T20:22:41.761855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:24.346362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749681+0000","last_change":"2024-04-26T20:22:41.749681+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749681+0000","last_undegraded":"2024-04-26T20:22:41.749681+0000","last_fullsized":"2024-04-26T20:22:41.749681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760841+0000","last_change":"2024-04-26T20:22:41.754578+0000","last_active":"2024-04-26T20:22:41.760841+0000","last_peered":"2024-04-26T20:22:41.760841+0000","last_clean":"2024-04-26T20:22:41.760841+0000","last_became_active":"2024-04-26T20:22:41.754286+0000","last_became_peered":"2024-04-26T20:22:41.754286+0000","last_unstale":"2024-04-26T20:22:41.760841+0000","last_undegraded":"2024-04-26T20:22:41.760841+0000","last_fullsized":"2024-04-26T20:22:41.760841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743070+0000","last_change":"2024-04-26T20:22:41.743070+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743070+0000","last_undegraded":"2024-04-26T20:22:41.743070+0000","last_fullsized":"2024-04-26T20:22:41.743070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765354+0000","last_change":"2024-04-26T20:22:41.762053+0000","last_active":"2024-04-26T20:22:41.765354+0000","last_peered":"2024-04-26T20:22:41.765354+0000","last_clean":"2024-04-26T20:22:41.765354+0000","last_became_active":"2024-04-26T20:22:41.761753+0000","last_became_peered":"2024-04-26T20:22:41.761753+0000","last_unstale":"2024-04-26T20:22:41.765354+0000","last_undegraded":"2024-04-26T20:22:41.765354+0000","last_fullsized":"2024-04-26T20:22:41.765354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:36.559062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748362+0000","last_change":"2024-04-26T20:22:41.748362+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748362+0000","last_undegraded":"2024-04-26T20:22:41.748362+0000","last_fullsized":"2024-04-26T20:22:41.748362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765262+0000","last_change":"2024-04-26T20:22:41.761862+0000","last_active":"2024-04-26T20:22:41.765262+0000","last_peered":"2024-04-26T20:22:41.765262+0000","last_clean":"2024-04-26T20:22:41.765262+0000","last_became_active":"2024-04-26T20:22:41.761611+0000","last_became_peered":"2024-04-26T20:22:41.761611+0000","last_unstale":"2024-04-26T20:22:41.765262+0000","last_undegraded":"2024-04-26T20:22:41.765262+0000","last_fullsized":"2024-04-26T20:22:41.765262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:02.047370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749794+0000","last_change":"2024-04-26T20:22:41.749794+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749794+0000","last_undegraded":"2024-04-26T20:22:41.749794+0000","last_fullsized":"2024-04-26T20:22:41.749794+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764613+0000","last_change":"2024-04-26T20:22:41.757671+0000","last_active":"2024-04-26T20:22:41.764613+0000","last_peered":"2024-04-26T20:22:41.764613+0000","last_clean":"2024-04-26T20:22:41.764613+0000","last_became_active":"2024-04-26T20:22:41.757277+0000","last_became_peered":"2024-04-26T20:22:41.757277+0000","last_unstale":"2024-04-26T20:22:41.764613+0000","last_undegraded":"2024-04-26T20:22:41.764613+0000","last_fullsized":"2024-04-26T20:22:41.764613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749475+0000","last_change":"2024-04-26T20:22:41.749475+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749475+0000","last_undegraded":"2024-04-26T20:22:41.749475+0000","last_fullsized":"2024-04-26T20:22:41.749475+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762294+0000","last_change":"2024-04-26T20:22:41.760284+0000","last_active":"2024-04-26T20:22:41.762294+0000","last_peered":"2024-04-26T20:22:41.762294+0000","last_clean":"2024-04-26T20:22:41.762294+0000","last_became_active":"2024-04-26T20:22:41.759981+0000","last_became_peered":"2024-04-26T20:22:41.759981+0000","last_unstale":"2024-04-26T20:22:41.762294+0000","last_undegraded":"2024-04-26T20:22:41.762294+0000","last_fullsized":"2024-04-26T20:22:41.762294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:20.612907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749667+0000","last_change":"2024-04-26T20:22:41.749667+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749667+0000","last_undegraded":"2024-04-26T20:22:41.749667+0000","last_fullsized":"2024-04-26T20:22:41.749667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764722+0000","last_change":"2024-04-26T20:22:41.759409+0000","last_active":"2024-04-26T20:22:41.764722+0000","last_peered":"2024-04-26T20:22:41.764722+0000","last_clean":"2024-04-26T20:22:41.764722+0000","last_became_active":"2024-04-26T20:22:41.758974+0000","last_became_peered":"2024-04-26T20:22:41.758974+0000","last_unstale":"2024-04-26T20:22:41.764722+0000","last_undegraded":"2024-04-26T20:22:41.764722+0000","last_fullsized":"2024-04-26T20:22:41.764722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:38.972810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743856+0000","last_change":"2024-04-26T20:22:41.743856+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743856+0000","last_undegraded":"2024-04-26T20:22:41.743856+0000","last_fullsized":"2024-04-26T20:22:41.743856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762234+0000","last_change":"2024-04-26T20:22:41.760955+0000","last_active":"2024-04-26T20:22:41.762234+0000","last_peered":"2024-04-26T20:22:41.762234+0000","last_clean":"2024-04-26T20:22:41.762234+0000","last_became_active":"2024-04-26T20:22:41.760658+0000","last_became_peered":"2024-04-26T20:22:41.760658+0000","last_unstale":"2024-04-26T20:22:41.762234+0000","last_undegraded":"2024-04-26T20:22:41.762234+0000","last_fullsized":"2024-04-26T20:22:41.762234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:42.223536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748548+0000","last_change":"2024-04-26T20:22:41.748548+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748548+0000","last_undegraded":"2024-04-26T20:22:41.748548+0000","last_fullsized":"2024-04-26T20:22:41.748548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758419+0000","last_change":"2024-04-26T20:22:41.751412+0000","last_active":"2024-04-26T20:22:41.758419+0000","last_peered":"2024-04-26T20:22:41.758419+0000","last_clean":"2024-04-26T20:22:41.758419+0000","last_became_active":"2024-04-26T20:22:41.751102+0000","last_became_peered":"2024-04-26T20:22:41.751102+0000","last_unstale":"2024-04-26T20:22:41.758419+0000","last_undegraded":"2024-04-26T20:22:41.758419+0000","last_fullsized":"2024-04-26T20:22:41.758419+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753116+0000","last_change":"2024-04-26T20:22:41.753116+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753116+0000","last_undegraded":"2024-04-26T20:22:41.753116+0000","last_fullsized":"2024-04-26T20:22:41.753116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765217+0000","last_change":"2024-04-26T20:22:41.760709+0000","last_active":"2024-04-26T20:22:41.765217+0000","last_peered":"2024-04-26T20:22:41.765217+0000","last_clean":"2024-04-26T20:22:41.765217+0000","last_became_active":"2024-04-26T20:22:41.760281+0000","last_became_peered":"2024-04-26T20:22:41.760281+0000","last_unstale":"2024-04-26T20:22:41.765217+0000","last_undegraded":"2024-04-26T20:22:41.765217+0000","last_fullsized":"2024-04-26T20:22:41.765217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:20.522370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745965+0000","last_change":"2024-04-26T20:22:41.745965+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745965+0000","last_undegraded":"2024-04-26T20:22:41.745965+0000","last_fullsized":"2024-04-26T20:22:41.745965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764753+0000","last_change":"2024-04-26T20:22:41.759620+0000","last_active":"2024-04-26T20:22:41.764753+0000","last_peered":"2024-04-26T20:22:41.764753+0000","last_clean":"2024-04-26T20:22:41.764753+0000","last_became_active":"2024-04-26T20:22:41.759154+0000","last_became_peered":"2024-04-26T20:22:41.759154+0000","last_unstale":"2024-04-26T20:22:41.764753+0000","last_undegraded":"2024-04-26T20:22:41.764753+0000","last_fullsized":"2024-04-26T20:22:41.764753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:42.413094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750914+0000","last_change":"2024-04-26T20:22:41.750914+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750914+0000","last_undegraded":"2024-04-26T20:22:41.750914+0000","last_fullsized":"2024-04-26T20:22:41.750914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761309+0000","last_change":"2024-04-26T20:22:41.758134+0000","last_active":"2024-04-26T20:22:41.761309+0000","last_peered":"2024-04-26T20:22:41.761309+0000","last_clean":"2024-04-26T20:22:41.761309+0000","last_became_active":"2024-04-26T20:22:41.757853+0000","last_became_peered":"2024-04-26T20:22:41.757853+0000","last_unstale":"2024-04-26T20:22:41.761309+0000","last_undegraded":"2024-04-26T20:22:41.761309+0000","last_fullsized":"2024-04-26T20:22:41.761309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.752554+0000","last_change":"2024-04-26T20:22:41.752554+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.752554+0000","last_undegraded":"2024-04-26T20:22:41.752554+0000","last_fullsized":"2024-04-26T20:22:41.752554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758900+0000","last_change":"2024-04-26T20:22:41.756567+0000","last_active":"2024-04-26T20:22:41.758900+0000","last_peered":"2024-04-26T20:22:41.758900+0000","last_clean":"2024-04-26T20:22:41.758900+0000","last_became_active":"2024-04-26T20:22:41.756259+0000","last_became_peered":"2024-04-26T20:22:41.756259+0000","last_unstale":"2024-04-26T20:22:41.758900+0000","last_undegraded":"2024-04-26T20:22:41.758900+0000","last_fullsized":"2024-04-26T20:22:41.758900+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:19.518020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745137+0000","last_change":"2024-04-26T20:22:41.745137+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745137+0000","last_undegraded":"2024-04-26T20:22:41.745137+0000","last_fullsized":"2024-04-26T20:22:41.745137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761886+0000","last_change":"2024-04-26T20:22:41.757636+0000","last_active":"2024-04-26T20:22:41.761886+0000","last_peered":"2024-04-26T20:22:41.761886+0000","last_clean":"2024-04-26T20:22:41.761886+0000","last_became_active":"2024-04-26T20:22:41.757329+0000","last_became_peered":"2024-04-26T20:22:41.757329+0000","last_unstale":"2024-04-26T20:22:41.761886+0000","last_undegraded":"2024-04-26T20:22:41.761886+0000","last_fullsized":"2024-04-26T20:22:41.761886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:38.757083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744642+0000","last_change":"2024-04-26T20:22:41.744642+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744642+0000","last_undegraded":"2024-04-26T20:22:41.744642+0000","last_fullsized":"2024-04-26T20:22:41.744642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762077+0000","last_change":"2024-04-26T20:22:41.759343+0000","last_active":"2024-04-26T20:22:41.762077+0000","last_peered":"2024-04-26T20:22:41.762077+0000","last_clean":"2024-04-26T20:22:41.762077+0000","last_became_active":"2024-04-26T20:22:41.758924+0000","last_became_peered":"2024-04-26T20:22:41.758924+0000","last_unstale":"2024-04-26T20:22:41.762077+0000","last_undegraded":"2024-04-26T20:22:41.762077+0000","last_fullsized":"2024-04-26T20:22:41.762077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.731025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743229+0000","last_change":"2024-04-26T20:22:41.743229+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743229+0000","last_undegraded":"2024-04-26T20:22:41.743229+0000","last_fullsized":"2024-04-26T20:22:41.743229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.749913+0000","last_change":"2024-04-26T20:22:41.747174+0000","last_active":"2024-04-26T20:22:41.749913+0000","last_peered":"2024-04-26T20:22:41.749913+0000","last_clean":"2024-04-26T20:22:41.749913+0000","last_became_active":"2024-04-26T20:22:41.746153+0000","last_became_peered":"2024-04-26T20:22:41.746153+0000","last_unstale":"2024-04-26T20:22:41.749913+0000","last_undegraded":"2024-04-26T20:22:41.749913+0000","last_fullsized":"2024-04-26T20:22:41.749913+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743176+0000","last_change":"2024-04-26T20:22:41.743176+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743176+0000","last_undegraded":"2024-04-26T20:22:41.743176+0000","last_fullsized":"2024-04-26T20:22:41.743176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760527+0000","last_change":"2024-04-26T20:22:41.755144+0000","last_active":"2024-04-26T20:22:41.760527+0000","last_peered":"2024-04-26T20:22:41.760527+0000","last_clean":"2024-04-26T20:22:41.760527+0000","last_became_active":"2024-04-26T20:22:41.754847+0000","last_became_peered":"2024-04-26T20:22:41.754847+0000","last_unstale":"2024-04-26T20:22:41.760527+0000","last_undegraded":"2024-04-26T20:22:41.760527+0000","last_fullsized":"2024-04-26T20:22:41.760527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745108+0000","last_change":"2024-04-26T20:22:41.745108+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745108+0000","last_undegraded":"2024-04-26T20:22:41.745108+0000","last_fullsized":"2024-04-26T20:22:41.745108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762202+0000","last_change":"2024-04-26T20:22:41.759321+0000","last_active":"2024-04-26T20:22:41.762202+0000","last_peered":"2024-04-26T20:22:41.762202+0000","last_clean":"2024-04-26T20:22:41.762202+0000","last_became_active":"2024-04-26T20:22:41.759034+0000","last_became_peered":"2024-04-26T20:22:41.759034+0000","last_unstale":"2024-04-26T20:22:41.762202+0000","last_undegraded":"2024-04-26T20:22:41.762202+0000","last_fullsized":"2024-04-26T20:22:41.762202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:19.607106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751655+0000","last_change":"2024-04-26T20:22:41.751655+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751655+0000","last_undegraded":"2024-04-26T20:22:41.751655+0000","last_fullsized":"2024-04-26T20:22:41.751655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761347+0000","last_change":"2024-04-26T20:22:41.760401+0000","last_active":"2024-04-26T20:22:41.761347+0000","last_peered":"2024-04-26T20:22:41.761347+0000","last_clean":"2024-04-26T20:22:41.761347+0000","last_became_active":"2024-04-26T20:22:41.760080+0000","last_became_peered":"2024-04-26T20:22:41.760080+0000","last_unstale":"2024-04-26T20:22:41.761347+0000","last_undegraded":"2024-04-26T20:22:41.761347+0000","last_fullsized":"2024-04-26T20:22:41.761347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:02.047370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744516+0000","last_change":"2024-04-26T20:22:41.744516+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744516+0000","last_undegraded":"2024-04-26T20:22:41.744516+0000","last_fullsized":"2024-04-26T20:22:41.744516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760372+0000","last_change":"2024-04-26T20:22:41.756976+0000","last_active":"2024-04-26T20:22:41.760372+0000","last_peered":"2024-04-26T20:22:41.760372+0000","last_clean":"2024-04-26T20:22:41.760372+0000","last_became_active":"2024-04-26T20:22:41.756639+0000","last_became_peered":"2024-04-26T20:22:41.756639+0000","last_unstale":"2024-04-26T20:22:41.760372+0000","last_undegraded":"2024-04-26T20:22:41.760372+0000","last_fullsized":"2024-04-26T20:22:41.760372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751992+0000","last_change":"2024-04-26T20:22:41.751992+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751992+0000","last_undegraded":"2024-04-26T20:22:41.751992+0000","last_fullsized":"2024-04-26T20:22:41.751992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.749878+0000","last_change":"2024-04-26T20:22:41.747756+0000","last_active":"2024-04-26T20:22:41.749878+0000","last_peered":"2024-04-26T20:22:41.749878+0000","last_clean":"2024-04-26T20:22:41.749878+0000","last_became_active":"2024-04-26T20:22:41.747460+0000","last_became_peered":"2024-04-26T20:22:41.747460+0000","last_unstale":"2024-04-26T20:22:41.749878+0000","last_undegraded":"2024-04-26T20:22:41.749878+0000","last_fullsized":"2024-04-26T20:22:41.749878+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.742836+0000","last_change":"2024-04-26T20:22:41.742836+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.742836+0000","last_undegraded":"2024-04-26T20:22:41.742836+0000","last_fullsized":"2024-04-26T20:22:41.742836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758269+0000","last_change":"2024-04-26T20:22:41.750670+0000","last_active":"2024-04-26T20:22:41.758269+0000","last_peered":"2024-04-26T20:22:41.758269+0000","last_clean":"2024-04-26T20:22:41.758269+0000","last_became_active":"2024-04-26T20:22:41.750317+0000","last_became_peered":"2024-04-26T20:22:41.750317+0000","last_unstale":"2024-04-26T20:22:41.758269+0000","last_undegraded":"2024-04-26T20:22:41.758269+0000","last_fullsized":"2024-04-26T20:22:41.758269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:55.932492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.747088+0000","last_change":"2024-04-26T20:22:41.747088+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.747088+0000","last_undegraded":"2024-04-26T20:22:41.747088+0000","last_fullsized":"2024-04-26T20:22:41.747088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.756385+0000","last_change":"2024-04-26T20:22:41.754948+0000","last_active":"2024-04-26T20:22:41.756385+0000","last_peered":"2024-04-26T20:22:41.756385+0000","last_clean":"2024-04-26T20:22:41.756385+0000","last_became_active":"2024-04-26T20:22:41.754664+0000","last_became_peered":"2024-04-26T20:22:41.754664+0000","last_unstale":"2024-04-26T20:22:41.756385+0000","last_undegraded":"2024-04-26T20:22:41.756385+0000","last_fullsized":"2024-04-26T20:22:41.756385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743378+0000","last_change":"2024-04-26T20:22:41.743378+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743378+0000","last_undegraded":"2024-04-26T20:22:41.743378+0000","last_fullsized":"2024-04-26T20:22:41.743378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760503+0000","last_change":"2024-04-26T20:22:41.756879+0000","last_active":"2024-04-26T20:22:41.760503+0000","last_peered":"2024-04-26T20:22:41.760503+0000","last_clean":"2024-04-26T20:22:41.760503+0000","last_became_active":"2024-04-26T20:22:41.756600+0000","last_became_peered":"2024-04-26T20:22:41.756600+0000","last_unstale":"2024-04-26T20:22:41.760503+0000","last_undegraded":"2024-04-26T20:22:41.760503+0000","last_fullsized":"2024-04-26T20:22:41.760503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750965+0000","last_change":"2024-04-26T20:22:41.750965+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750965+0000","last_undegraded":"2024-04-26T20:22:41.750965+0000","last_fullsized":"2024-04-26T20:22:41.750965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.759460+0000","last_change":"2024-04-26T20:22:41.756827+0000","last_active":"2024-04-26T20:22:41.759460+0000","last_peered":"2024-04-26T20:22:41.759460+0000","last_clean":"2024-04-26T20:22:41.759460+0000","last_became_active":"2024-04-26T20:22:41.756542+0000","last_became_peered":"2024-04-26T20:22:41.756542+0000","last_unstale":"2024-04-26T20:22:41.759460+0000","last_undegraded":"2024-04-26T20:22:41.759460+0000","last_fullsized":"2024-04-26T20:22:41.759460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:52.253630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748484+0000","last_change":"2024-04-26T20:22:41.748484+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748484+0000","last_undegraded":"2024-04-26T20:22:41.748484+0000","last_fullsized":"2024-04-26T20:22:41.748484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765234+0000","last_change":"2024-04-26T20:22:41.760981+0000","last_active":"2024-04-26T20:22:41.765234+0000","last_peered":"2024-04-26T20:22:41.765234+0000","last_clean":"2024-04-26T20:22:41.765234+0000","last_became_active":"2024-04-26T20:22:41.760700+0000","last_became_peered":"2024-04-26T20:22:41.760700+0000","last_unstale":"2024-04-26T20:22:41.765234+0000","last_undegraded":"2024-04-26T20:22:41.765234+0000","last_fullsized":"2024-04-26T20:22:41.765234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:20.612907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750327+0000","last_change":"2024-04-26T20:22:41.750327+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750327+0000","last_undegraded":"2024-04-26T20:22:41.750327+0000","last_fullsized":"2024-04-26T20:22:41.750327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760640+0000","last_change":"2024-04-26T20:22:41.755373+0000","last_active":"2024-04-26T20:22:41.760640+0000","last_peered":"2024-04-26T20:22:41.760640+0000","last_clean":"2024-04-26T20:22:41.760640+0000","last_became_active":"2024-04-26T20:22:41.755087+0000","last_became_peered":"2024-04-26T20:22:41.755087+0000","last_unstale":"2024-04-26T20:22:41.760640+0000","last_undegraded":"2024-04-26T20:22:41.760640+0000","last_fullsized":"2024-04-26T20:22:41.760640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.756741+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:22:41.756741+0000","last_peered":"2024-04-26T20:22:41.756741+0000","last_clean":"2024-04-26T20:22:41.756741+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:22:41.756741+0000","last_undegraded":"2024-04-26T20:22:41.756741+0000","last_fullsized":"2024-04-26T20:22:41.756741+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:22:46.774 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:22:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.910+0000 7f64efab1640 1 Processor -- start 2024-04-26T20:22:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.910+0000 7f64efab1640 1 -- start start 2024-04-26T20:22:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.911+0000 7f64efab1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f64e80cff40 0x7f64e80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.911+0000 7f64efab1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e80b1d70 con 0x7f64e8162e40 2024-04-26T20:22:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.911+0000 7f64efab1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e80d1960 con 0x7f64e806ea40 2024-04-26T20:22:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.911+0000 7f64efab1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f64e8163220 con 0x7f64e80cff40 2024-04-26T20:22:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.911+0000 7f64ee04e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f64e80cff40 0x7f64e80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.911+0000 7f64ed84d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f64e8162e40 0x7f64e80b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52570/0 (socket says 172.21.15.37:52570) 2024-04-26T20:22:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.911+0000 7f64ed84d640 1 -- 172.21.15.37:0/2748895532 learned_addr learned my addr 172.21.15.37:0/2748895532 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.911+0000 7f64ee04e640 1 -- 172.21.15.37:0/2748895532 >> v1:172.21.15.134:6789/0 conn(0x7f64e806ea40 legacy=0x7f64e80cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.911+0000 7f64ee04e640 1 -- 172.21.15.37:0/2748895532 >> v1:172.21.15.37:6789/0 conn(0x7f64e8162e40 legacy=0x7f64e80b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.911+0000 7f64ee04e640 1 -- 172.21.15.37:0/2748895532 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e80d1c40 con 0x7f64e80cff40 2024-04-26T20:22:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.912+0000 7f64ee04e640 1 --2- 172.21.15.37:0/2748895532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e80cff40 0x7f64e80d1340 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009d90 tx=0x7f64e002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0bb7b550665ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.912+0000 7f64ec84b640 1 -- 172.21.15.37:0/2748895532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0031600 con 0x7f64e80cff40 2024-04-26T20:22:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.912+0000 7f64ec84b640 1 -- 172.21.15.37:0/2748895532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e00317a0 con 0x7f64e80cff40 2024-04-26T20:22:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.912+0000 7f64ec84b640 1 -- 172.21.15.37:0/2748895532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0031b70 con 0x7f64e80cff40 2024-04-26T20:22:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.913+0000 7f64efab1640 1 -- 172.21.15.37:0/2748895532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e80cff40 msgr2=0x7f64e80d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.913+0000 7f64efab1640 1 --2- 172.21.15.37:0/2748895532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e80cff40 0x7f64e80d1340 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009d90 tx=0x7f64e002a610 comp rx=0 tx=0).stop 2024-04-26T20:22:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.913+0000 7f64efab1640 1 -- 172.21.15.37:0/2748895532 shutdown_connections 2024-04-26T20:22:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.913+0000 7f64efab1640 1 --2- 172.21.15.37:0/2748895532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e80cff40 0x7f64e80d1340 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.913+0000 7f64efab1640 1 -- 172.21.15.37:0/2748895532 >> 172.21.15.37:0/2748895532 conn(0x7f64e809ba50 msgr2=0x7f64e809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.913+0000 7f64efab1640 1 -- 172.21.15.37:0/2748895532 shutdown_connections 2024-04-26T20:22:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.913+0000 7f64efab1640 1 -- 172.21.15.37:0/2748895532 wait complete. 2024-04-26T20:22:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.914+0000 7f64efab1640 1 Processor -- start 2024-04-26T20:22:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.914+0000 7f64efab1640 1 -- start start 2024-04-26T20:22:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64efab1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e806ea40 0x7f64e80c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64efab1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e80cff40 0x7f64e81f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64efab1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64e8162e40 0x7f64e80b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64efab1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f64e80ce010 con 0x7f64e806ea40 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64efab1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64e80d0eb0 con 0x7f64e80cff40 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64efab1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f64e80ce190 con 0x7f64e8162e40 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64ed84d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e806ea40 0x7f64e80c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64ed84d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e806ea40 0x7f64e80c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35840/0 (socket says 172.21.15.37:35840) 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64ed84d640 1 -- 172.21.15.37:0/1725003697 learned_addr learned my addr 172.21.15.37:0/1725003697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64ed04c640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e80cff40 0x7f64e81f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64ee04e640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64e8162e40 0x7f64e80b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64ed84d640 1 -- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64e8162e40 msgr2=0x7f64e80b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64ed84d640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64e8162e40 0x7f64e80b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64ed84d640 1 -- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e80cff40 msgr2=0x7f64e81f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64ed84d640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e80cff40 0x7f64e81f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64ed84d640 1 -- 172.21.15.37:0/1725003697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e81f5c50 con 0x7f64e806ea40 2024-04-26T20:22:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64ed84d640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e806ea40 0x7f64e80c7b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c3c0 tx=0x7f64e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64de7fc640 1 -- 172.21.15.37:0/1725003697 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4017070 con 0x7f64e806ea40 2024-04-26T20:22:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64de7fc640 1 -- 172.21.15.37:0/1725003697 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e4005ce0 con 0x7f64e806ea40 2024-04-26T20:22:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64efab1640 1 -- 172.21.15.37:0/1725003697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e81f5f30 con 0x7f64e806ea40 2024-04-26T20:22:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.915+0000 7f64de7fc640 1 -- 172.21.15.37:0/1725003697 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e40072f0 con 0x7f64e806ea40 2024-04-26T20:22:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.916+0000 7f64efab1640 1 -- 172.21.15.37:0/1725003697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e815fa10 con 0x7f64e806ea40 2024-04-26T20:22:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.917+0000 7f64efab1640 1 -- 172.21.15.37:0/1725003697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e806a360 con 0x7f64e806ea40 2024-04-26T20:22:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.917+0000 7f64de7fc640 1 -- 172.21.15.37:0/1725003697 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64e401a070 con 0x7f64e806ea40 2024-04-26T20:22:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.917+0000 7f64de7fc640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f64cc06c8f0 0x7f64cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.917+0000 7f64de7fc640 1 -- 172.21.15.37:0/1725003697 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f64e408aea0 con 0x7f64e806ea40 2024-04-26T20:22:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.918+0000 7f64ed04c640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f64cc06c8f0 0x7f64cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.918+0000 7f64ed04c640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f64cc06c8f0 0x7f64cc06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f64e80c7940 tx=0x7f64d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:46.922+0000 7f64de7fc640 1 -- 172.21.15.37:0/1725003697 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64e40592d0 con 0x7f64e806ea40 2024-04-26T20:22:47.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.101+0000 7f64efab1640 1 -- 172.21.15.37:0/1725003697 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64e815fec0 con 0x7f64cc06c8f0 2024-04-26T20:22:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.108+0000 7f64de7fc640 1 -- 172.21.15.37:0/1725003697 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7f64e815fec0 con 0x7f64cc06c8f0 2024-04-26T20:22:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:47.113 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 -- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f64cc06c8f0 msgr2=0x7f64cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f64cc06c8f0 0x7f64cc06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f64e80c7940 tx=0x7f64d8009040 comp rx=0 tx=0).stop 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 -- 172.21.15.37:0/1725003697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e806ea40 msgr2=0x7f64e80c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e806ea40 0x7f64e80c7b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c3c0 tx=0x7f64e400c880 comp rx=0 tx=0).stop 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 -- 172.21.15.37:0/1725003697 shutdown_connections 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f64cc06c8f0 0x7f64cc06edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64e8162e40 0x7f64e80b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e80cff40 0x7f64e81f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 --2- 172.21.15.37:0/1725003697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e806ea40 0x7f64e80c7b50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 -- 172.21.15.37:0/1725003697 >> 172.21.15.37:0/1725003697 conn(0x7f64e809ba50 msgr2=0x7f64e8160280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 -- 172.21.15.37:0/1725003697 shutdown_connections 2024-04-26T20:22:47.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.111+0000 7f64efab1640 1 -- 172.21.15.37:0/1725003697 wait complete. 2024-04-26T20:22:47.123 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-04-26T20:22:45.346702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220008,"kb_used_data":3832,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754712,"statfs":{"total":773094113280,"available":772868825088,"internally_reserved":0,"allocated":3923968,"data_stored":2223996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.613063"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.747315+0000","last_change":"2024-04-26T20:22:41.747315+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.747315+0000","last_undegraded":"2024-04-26T20:22:41.747315+0000","last_fullsized":"2024-04-26T20:22:41.747315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751176+0000","last_change":"2024-04-26T20:22:41.751176+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751176+0000","last_undegraded":"2024-04-26T20:22:41.751176+0000","last_fullsized":"2024-04-26T20:22:41.751176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749201+0000","last_change":"2024-04-26T20:22:41.749201+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749201+0000","last_undegraded":"2024-04-26T20:22:41.749201+0000","last_fullsized":"2024-04-26T20:22:41.749201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745545+0000","last_change":"2024-04-26T20:22:41.745545+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745545+0000","last_undegraded":"2024-04-26T20:22:41.745545+0000","last_fullsized":"2024-04-26T20:22:41.745545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745404+0000","last_change":"2024-04-26T20:22:41.745404+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745404+0000","last_undegraded":"2024-04-26T20:22:41.745404+0000","last_fullsized":"2024-04-26T20:22:41.745404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750104+0000","last_change":"2024-04-26T20:22:41.750104+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750104+0000","last_undegraded":"2024-04-26T20:22:41.750104+0000","last_fullsized":"2024-04-26T20:22:41.750104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750969+0000","last_change":"2024-04-26T20:22:41.750969+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750969+0000","last_undegraded":"2024-04-26T20:22:41.750969+0000","last_fullsized":"2024-04-26T20:22:41.750969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753949+0000","last_change":"2024-04-26T20:22:41.753949+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753949+0000","last_undegraded":"2024-04-26T20:22:41.753949+0000","last_fullsized":"2024-04-26T20:22:41.753949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749215+0000","last_change":"2024-04-26T20:22:41.749215+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749215+0000","last_undegraded":"2024-04-26T20:22:41.749215+0000","last_fullsized":"2024-04-26T20:22:41.749215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744734+0000","last_change":"2024-04-26T20:22:41.744734+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744734+0000","last_undegraded":"2024-04-26T20:22:41.744734+0000","last_fullsized":"2024-04-26T20:22:41.744734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751008+0000","last_change":"2024-04-26T20:22:41.751008+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751008+0000","last_undegraded":"2024-04-26T20:22:41.751008+0000","last_fullsized":"2024-04-26T20:22:41.751008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751513+0000","last_change":"2024-04-26T20:22:41.751513+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751513+0000","last_undegraded":"2024-04-26T20:22:41.751513+0000","last_fullsized":"2024-04-26T20:22:41.751513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744914+0000","last_change":"2024-04-26T20:22:41.744914+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744914+0000","last_undegraded":"2024-04-26T20:22:41.744914+0000","last_fullsized":"2024-04-26T20:22:41.744914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749030+0000","last_change":"2024-04-26T20:22:41.749030+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749030+0000","last_undegraded":"2024-04-26T20:22:41.749030+0000","last_fullsized":"2024-04-26T20:22:41.749030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750437+0000","last_change":"2024-04-26T20:22:41.750437+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750437+0000","last_undegraded":"2024-04-26T20:22:41.750437+0000","last_fullsized":"2024-04-26T20:22:41.750437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751270+0000","last_change":"2024-04-26T20:22:41.751270+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751270+0000","last_undegraded":"2024-04-26T20:22:41.751270+0000","last_fullsized":"2024-04-26T20:22:41.751270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751848+0000","last_change":"2024-04-26T20:22:41.751848+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751848+0000","last_undegraded":"2024-04-26T20:22:41.751848+0000","last_fullsized":"2024-04-26T20:22:41.751848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750736+0000","last_change":"2024-04-26T20:22:41.750736+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750736+0000","last_undegraded":"2024-04-26T20:22:41.750736+0000","last_fullsized":"2024-04-26T20:22:41.750736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750525+0000","last_change":"2024-04-26T20:22:41.750525+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750525+0000","last_undegraded":"2024-04-26T20:22:41.750525+0000","last_fullsized":"2024-04-26T20:22:41.750525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744169+0000","last_change":"2024-04-26T20:22:41.744169+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744169+0000","last_undegraded":"2024-04-26T20:22:41.744169+0000","last_fullsized":"2024-04-26T20:22:41.744169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748990+0000","last_change":"2024-04-26T20:22:41.748990+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748990+0000","last_undegraded":"2024-04-26T20:22:41.748990+0000","last_fullsized":"2024-04-26T20:22:41.748990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750071+0000","last_change":"2024-04-26T20:22:41.750071+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750071+0000","last_undegraded":"2024-04-26T20:22:41.750071+0000","last_fullsized":"2024-04-26T20:22:41.750071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744336+0000","last_change":"2024-04-26T20:22:41.744336+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744336+0000","last_undegraded":"2024-04-26T20:22:41.744336+0000","last_fullsized":"2024-04-26T20:22:41.744336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749073+0000","last_change":"2024-04-26T20:22:41.749073+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749073+0000","last_undegraded":"2024-04-26T20:22:41.749073+0000","last_fullsized":"2024-04-26T20:22:41.749073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745963+0000","last_change":"2024-04-26T20:22:41.745963+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745963+0000","last_undegraded":"2024-04-26T20:22:41.745963+0000","last_fullsized":"2024-04-26T20:22:41.745963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750267+0000","last_change":"2024-04-26T20:22:41.750267+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750267+0000","last_undegraded":"2024-04-26T20:22:41.750267+0000","last_fullsized":"2024-04-26T20:22:41.750267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744351+0000","last_change":"2024-04-26T20:22:41.744351+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744351+0000","last_undegraded":"2024-04-26T20:22:41.744351+0000","last_fullsized":"2024-04-26T20:22:41.744351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748823+0000","last_change":"2024-04-26T20:22:41.748823+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748823+0000","last_undegraded":"2024-04-26T20:22:41.748823+0000","last_fullsized":"2024-04-26T20:22:41.748823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748525+0000","last_change":"2024-04-26T20:22:41.748525+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748525+0000","last_undegraded":"2024-04-26T20:22:41.748525+0000","last_fullsized":"2024-04-26T20:22:41.748525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.752597+0000","last_change":"2024-04-26T20:22:41.752597+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.752597+0000","last_undegraded":"2024-04-26T20:22:41.752597+0000","last_fullsized":"2024-04-26T20:22:41.752597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.746522+0000","last_change":"2024-04-26T20:22:41.746522+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.746522+0000","last_undegraded":"2024-04-26T20:22:41.746522+0000","last_fullsized":"2024-04-26T20:22:41.746522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753519+0000","last_change":"2024-04-26T20:22:41.753519+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753519+0000","last_undegraded":"2024-04-26T20:22:41.753519+0000","last_fullsized":"2024-04-26T20:22:41.753519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744224+0000","last_change":"2024-04-26T20:22:41.744224+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744224+0000","last_undegraded":"2024-04-26T20:22:41.744224+0000","last_fullsized":"2024-04-26T20:22:41.744224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760557+0000","last_change":"2024-04-26T20:22:41.756667+0000","last_active":"2024-04-26T20:22:41.760557+0000","last_peered":"2024-04-26T20:22:41.760557+0000","last_clean":"2024-04-26T20:22:41.760557+0000","last_became_active":"2024-04-26T20:22:41.756385+0000","last_became_peered":"2024-04-26T20:22:41.756385+0000","last_unstale":"2024-04-26T20:22:41.760557+0000","last_undegraded":"2024-04-26T20:22:41.760557+0000","last_fullsized":"2024-04-26T20:22:41.760557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.746845+0000","last_change":"2024-04-26T20:22:41.746845+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.746845+0000","last_undegraded":"2024-04-26T20:22:41.746845+0000","last_fullsized":"2024-04-26T20:22:41.746845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764642+0000","last_change":"2024-04-26T20:22:41.758415+0000","last_active":"2024-04-26T20:22:41.764642+0000","last_peered":"2024-04-26T20:22:41.764642+0000","last_clean":"2024-04-26T20:22:41.764642+0000","last_became_active":"2024-04-26T20:22:41.757712+0000","last_became_peered":"2024-04-26T20:22:41.757712+0000","last_unstale":"2024-04-26T20:22:41.764642+0000","last_undegraded":"2024-04-26T20:22:41.764642+0000","last_fullsized":"2024-04-26T20:22:41.764642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:38.972810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749597+0000","last_change":"2024-04-26T20:22:41.749597+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749597+0000","last_undegraded":"2024-04-26T20:22:41.749597+0000","last_fullsized":"2024-04-26T20:22:41.749597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764752+0000","last_change":"2024-04-26T20:22:41.759486+0000","last_active":"2024-04-26T20:22:41.764752+0000","last_peered":"2024-04-26T20:22:41.764752+0000","last_clean":"2024-04-26T20:22:41.764752+0000","last_became_active":"2024-04-26T20:22:41.759076+0000","last_became_peered":"2024-04-26T20:22:41.759076+0000","last_unstale":"2024-04-26T20:22:41.764752+0000","last_undegraded":"2024-04-26T20:22:41.764752+0000","last_fullsized":"2024-04-26T20:22:41.764752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:27.337131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743760+0000","last_change":"2024-04-26T20:22:41.743760+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743760+0000","last_undegraded":"2024-04-26T20:22:41.743760+0000","last_fullsized":"2024-04-26T20:22:41.743760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758241+0000","last_change":"2024-04-26T20:22:41.754766+0000","last_active":"2024-04-26T20:22:41.758241+0000","last_peered":"2024-04-26T20:22:41.758241+0000","last_clean":"2024-04-26T20:22:41.758241+0000","last_became_active":"2024-04-26T20:22:41.754427+0000","last_became_peered":"2024-04-26T20:22:41.754427+0000","last_unstale":"2024-04-26T20:22:41.758241+0000","last_undegraded":"2024-04-26T20:22:41.758241+0000","last_fullsized":"2024-04-26T20:22:41.758241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:52.253630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750686+0000","last_change":"2024-04-26T20:22:41.750686+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750686+0000","last_undegraded":"2024-04-26T20:22:41.750686+0000","last_fullsized":"2024-04-26T20:22:41.750686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762211+0000","last_change":"2024-04-26T20:22:41.759951+0000","last_active":"2024-04-26T20:22:41.762211+0000","last_peered":"2024-04-26T20:22:41.762211+0000","last_clean":"2024-04-26T20:22:41.762211+0000","last_became_active":"2024-04-26T20:22:41.759681+0000","last_became_peered":"2024-04-26T20:22:41.759681+0000","last_unstale":"2024-04-26T20:22:41.762211+0000","last_undegraded":"2024-04-26T20:22:41.762211+0000","last_fullsized":"2024-04-26T20:22:41.762211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:49.082266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750653+0000","last_change":"2024-04-26T20:22:41.750653+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750653+0000","last_undegraded":"2024-04-26T20:22:41.750653+0000","last_fullsized":"2024-04-26T20:22:41.750653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761855+0000","last_change":"2024-04-26T20:22:41.758461+0000","last_active":"2024-04-26T20:22:41.761855+0000","last_peered":"2024-04-26T20:22:41.761855+0000","last_clean":"2024-04-26T20:22:41.761855+0000","last_became_active":"2024-04-26T20:22:41.758166+0000","last_became_peered":"2024-04-26T20:22:41.758166+0000","last_unstale":"2024-04-26T20:22:41.761855+0000","last_undegraded":"2024-04-26T20:22:41.761855+0000","last_fullsized":"2024-04-26T20:22:41.761855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:24.346362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749681+0000","last_change":"2024-04-26T20:22:41.749681+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749681+0000","last_undegraded":"2024-04-26T20:22:41.749681+0000","last_fullsized":"2024-04-26T20:22:41.749681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760841+0000","last_change":"2024-04-26T20:22:41.754578+0000","last_active":"2024-04-26T20:22:41.760841+0000","last_peered":"2024-04-26T20:22:41.760841+0000","last_clean":"2024-04-26T20:22:41.760841+0000","last_became_active":"2024-04-26T20:22:41.754286+0000","last_became_peered":"2024-04-26T20:22:41.754286+0000","last_unstale":"2024-04-26T20:22:41.760841+0000","last_undegraded":"2024-04-26T20:22:41.760841+0000","last_fullsized":"2024-04-26T20:22:41.760841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743070+0000","last_change":"2024-04-26T20:22:41.743070+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743070+0000","last_undegraded":"2024-04-26T20:22:41.743070+0000","last_fullsized":"2024-04-26T20:22:41.743070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765354+0000","last_change":"2024-04-26T20:22:41.762053+0000","last_active":"2024-04-26T20:22:41.765354+0000","last_peered":"2024-04-26T20:22:41.765354+0000","last_clean":"2024-04-26T20:22:41.765354+0000","last_became_active":"2024-04-26T20:22:41.761753+0000","last_became_peered":"2024-04-26T20:22:41.761753+0000","last_unstale":"2024-04-26T20:22:41.765354+0000","last_undegraded":"2024-04-26T20:22:41.765354+0000","last_fullsized":"2024-04-26T20:22:41.765354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:36.559062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748362+0000","last_change":"2024-04-26T20:22:41.748362+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748362+0000","last_undegraded":"2024-04-26T20:22:41.748362+0000","last_fullsized":"2024-04-26T20:22:41.748362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765262+0000","last_change":"2024-04-26T20:22:41.761862+0000","last_active":"2024-04-26T20:22:41.765262+0000","last_peered":"2024-04-26T20:22:41.765262+0000","last_clean":"2024-04-26T20:22:41.765262+0000","last_became_active":"2024-04-26T20:22:41.761611+0000","last_became_peered":"2024-04-26T20:22:41.761611+0000","last_unstale":"2024-04-26T20:22:41.765262+0000","last_undegraded":"2024-04-26T20:22:41.765262+0000","last_fullsized":"2024-04-26T20:22:41.765262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:02.047370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749794+0000","last_change":"2024-04-26T20:22:41.749794+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749794+0000","last_undegraded":"2024-04-26T20:22:41.749794+0000","last_fullsized":"2024-04-26T20:22:41.749794+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764613+0000","last_change":"2024-04-26T20:22:41.757671+0000","last_active":"2024-04-26T20:22:41.764613+0000","last_peered":"2024-04-26T20:22:41.764613+0000","last_clean":"2024-04-26T20:22:41.764613+0000","last_became_active":"2024-04-26T20:22:41.757277+0000","last_became_peered":"2024-04-26T20:22:41.757277+0000","last_unstale":"2024-04-26T20:22:41.764613+0000","last_undegraded":"2024-04-26T20:22:41.764613+0000","last_fullsized":"2024-04-26T20:22:41.764613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749475+0000","last_change":"2024-04-26T20:22:41.749475+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749475+0000","last_undegraded":"2024-04-26T20:22:41.749475+0000","last_fullsized":"2024-04-26T20:22:41.749475+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762294+0000","last_change":"2024-04-26T20:22:41.760284+0000","last_active":"2024-04-26T20:22:41.762294+0000","last_peered":"2024-04-26T20:22:41.762294+0000","last_clean":"2024-04-26T20:22:41.762294+0000","last_became_active":"2024-04-26T20:22:41.759981+0000","last_became_peered":"2024-04-26T20:22:41.759981+0000","last_unstale":"2024-04-26T20:22:41.762294+0000","last_undegraded":"2024-04-26T20:22:41.762294+0000","last_fullsized":"2024-04-26T20:22:41.762294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:20.612907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749667+0000","last_change":"2024-04-26T20:22:41.749667+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749667+0000","last_undegraded":"2024-04-26T20:22:41.749667+0000","last_fullsized":"2024-04-26T20:22:41.749667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764722+0000","last_change":"2024-04-26T20:22:41.759409+0000","last_active":"2024-04-26T20:22:41.764722+0000","last_peered":"2024-04-26T20:22:41.764722+0000","last_clean":"2024-04-26T20:22:41.764722+0000","last_became_active":"2024-04-26T20:22:41.758974+0000","last_became_peered":"2024-04-26T20:22:41.758974+0000","last_unstale":"2024-04-26T20:22:41.764722+0000","last_undegraded":"2024-04-26T20:22:41.764722+0000","last_fullsized":"2024-04-26T20:22:41.764722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:38.972810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743856+0000","last_change":"2024-04-26T20:22:41.743856+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743856+0000","last_undegraded":"2024-04-26T20:22:41.743856+0000","last_fullsized":"2024-04-26T20:22:41.743856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762234+0000","last_change":"2024-04-26T20:22:41.760955+0000","last_active":"2024-04-26T20:22:41.762234+0000","last_peered":"2024-04-26T20:22:41.762234+0000","last_clean":"2024-04-26T20:22:41.762234+0000","last_became_active":"2024-04-26T20:22:41.760658+0000","last_became_peered":"2024-04-26T20:22:41.760658+0000","last_unstale":"2024-04-26T20:22:41.762234+0000","last_undegraded":"2024-04-26T20:22:41.762234+0000","last_fullsized":"2024-04-26T20:22:41.762234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:42.223536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748548+0000","last_change":"2024-04-26T20:22:41.748548+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748548+0000","last_undegraded":"2024-04-26T20:22:41.748548+0000","last_fullsized":"2024-04-26T20:22:41.748548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758419+0000","last_change":"2024-04-26T20:22:41.751412+0000","last_active":"2024-04-26T20:22:41.758419+0000","last_peered":"2024-04-26T20:22:41.758419+0000","last_clean":"2024-04-26T20:22:41.758419+0000","last_became_active":"2024-04-26T20:22:41.751102+0000","last_became_peered":"2024-04-26T20:22:41.751102+0000","last_unstale":"2024-04-26T20:22:41.758419+0000","last_undegraded":"2024-04-26T20:22:41.758419+0000","last_fullsized":"2024-04-26T20:22:41.758419+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753116+0000","last_change":"2024-04-26T20:22:41.753116+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753116+0000","last_undegraded":"2024-04-26T20:22:41.753116+0000","last_fullsized":"2024-04-26T20:22:41.753116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765217+0000","last_change":"2024-04-26T20:22:41.760709+0000","last_active":"2024-04-26T20:22:41.765217+0000","last_peered":"2024-04-26T20:22:41.765217+0000","last_clean":"2024-04-26T20:22:41.765217+0000","last_became_active":"2024-04-26T20:22:41.760281+0000","last_became_peered":"2024-04-26T20:22:41.760281+0000","last_unstale":"2024-04-26T20:22:41.765217+0000","last_undegraded":"2024-04-26T20:22:41.765217+0000","last_fullsized":"2024-04-26T20:22:41.765217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:20.522370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745965+0000","last_change":"2024-04-26T20:22:41.745965+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745965+0000","last_undegraded":"2024-04-26T20:22:41.745965+0000","last_fullsized":"2024-04-26T20:22:41.745965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764753+0000","last_change":"2024-04-26T20:22:41.759620+0000","last_active":"2024-04-26T20:22:41.764753+0000","last_peered":"2024-04-26T20:22:41.764753+0000","last_clean":"2024-04-26T20:22:41.764753+0000","last_became_active":"2024-04-26T20:22:41.759154+0000","last_became_peered":"2024-04-26T20:22:41.759154+0000","last_unstale":"2024-04-26T20:22:41.764753+0000","last_undegraded":"2024-04-26T20:22:41.764753+0000","last_fullsized":"2024-04-26T20:22:41.764753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:42.413094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750914+0000","last_change":"2024-04-26T20:22:41.750914+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750914+0000","last_undegraded":"2024-04-26T20:22:41.750914+0000","last_fullsized":"2024-04-26T20:22:41.750914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761309+0000","last_change":"2024-04-26T20:22:41.758134+0000","last_active":"2024-04-26T20:22:41.761309+0000","last_peered":"2024-04-26T20:22:41.761309+0000","last_clean":"2024-04-26T20:22:41.761309+0000","last_became_active":"2024-04-26T20:22:41.757853+0000","last_became_peered":"2024-04-26T20:22:41.757853+0000","last_unstale":"2024-04-26T20:22:41.761309+0000","last_undegraded":"2024-04-26T20:22:41.761309+0000","last_fullsized":"2024-04-26T20:22:41.761309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.752554+0000","last_change":"2024-04-26T20:22:41.752554+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.752554+0000","last_undegraded":"2024-04-26T20:22:41.752554+0000","last_fullsized":"2024-04-26T20:22:41.752554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758900+0000","last_change":"2024-04-26T20:22:41.756567+0000","last_active":"2024-04-26T20:22:41.758900+0000","last_peered":"2024-04-26T20:22:41.758900+0000","last_clean":"2024-04-26T20:22:41.758900+0000","last_became_active":"2024-04-26T20:22:41.756259+0000","last_became_peered":"2024-04-26T20:22:41.756259+0000","last_unstale":"2024-04-26T20:22:41.758900+0000","last_undegraded":"2024-04-26T20:22:41.758900+0000","last_fullsized":"2024-04-26T20:22:41.758900+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:19.518020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745137+0000","last_change":"2024-04-26T20:22:41.745137+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745137+0000","last_undegraded":"2024-04-26T20:22:41.745137+0000","last_fullsized":"2024-04-26T20:22:41.745137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761886+0000","last_change":"2024-04-26T20:22:41.757636+0000","last_active":"2024-04-26T20:22:41.761886+0000","last_peered":"2024-04-26T20:22:41.761886+0000","last_clean":"2024-04-26T20:22:41.761886+0000","last_became_active":"2024-04-26T20:22:41.757329+0000","last_became_peered":"2024-04-26T20:22:41.757329+0000","last_unstale":"2024-04-26T20:22:41.761886+0000","last_undegraded":"2024-04-26T20:22:41.761886+0000","last_fullsized":"2024-04-26T20:22:41.761886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:38.757083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744642+0000","last_change":"2024-04-26T20:22:41.744642+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744642+0000","last_undegraded":"2024-04-26T20:22:41.744642+0000","last_fullsized":"2024-04-26T20:22:41.744642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762077+0000","last_change":"2024-04-26T20:22:41.759343+0000","last_active":"2024-04-26T20:22:41.762077+0000","last_peered":"2024-04-26T20:22:41.762077+0000","last_clean":"2024-04-26T20:22:41.762077+0000","last_became_active":"2024-04-26T20:22:41.758924+0000","last_became_peered":"2024-04-26T20:22:41.758924+0000","last_unstale":"2024-04-26T20:22:41.762077+0000","last_undegraded":"2024-04-26T20:22:41.762077+0000","last_fullsized":"2024-04-26T20:22:41.762077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.731025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743229+0000","last_change":"2024-04-26T20:22:41.743229+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743229+0000","last_undegraded":"2024-04-26T20:22:41.743229+0000","last_fullsized":"2024-04-26T20:22:41.743229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.749913+0000","last_change":"2024-04-26T20:22:41.747174+0000","last_active":"2024-04-26T20:22:41.749913+0000","last_peered":"2024-04-26T20:22:41.749913+0000","last_clean":"2024-04-26T20:22:41.749913+0000","last_became_active":"2024-04-26T20:22:41.746153+0000","last_became_peered":"2024-04-26T20:22:41.746153+0000","last_unstale":"2024-04-26T20:22:41.749913+0000","last_undegraded":"2024-04-26T20:22:41.749913+0000","last_fullsized":"2024-04-26T20:22:41.749913+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743176+0000","last_change":"2024-04-26T20:22:41.743176+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743176+0000","last_undegraded":"2024-04-26T20:22:41.743176+0000","last_fullsized":"2024-04-26T20:22:41.743176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760527+0000","last_change":"2024-04-26T20:22:41.755144+0000","last_active":"2024-04-26T20:22:41.760527+0000","last_peered":"2024-04-26T20:22:41.760527+0000","last_clean":"2024-04-26T20:22:41.760527+0000","last_became_active":"2024-04-26T20:22:41.754847+0000","last_became_peered":"2024-04-26T20:22:41.754847+0000","last_unstale":"2024-04-26T20:22:41.760527+0000","last_undegraded":"2024-04-26T20:22:41.760527+0000","last_fullsized":"2024-04-26T20:22:41.760527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745108+0000","last_change":"2024-04-26T20:22:41.745108+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745108+0000","last_undegraded":"2024-04-26T20:22:41.745108+0000","last_fullsized":"2024-04-26T20:22:41.745108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762202+0000","last_change":"2024-04-26T20:22:41.759321+0000","last_active":"2024-04-26T20:22:41.762202+0000","last_peered":"2024-04-26T20:22:41.762202+0000","last_clean":"2024-04-26T20:22:41.762202+0000","last_became_active":"2024-04-26T20:22:41.759034+0000","last_became_peered":"2024-04-26T20:22:41.759034+0000","last_unstale":"2024-04-26T20:22:41.762202+0000","last_undegraded":"2024-04-26T20:22:41.762202+0000","last_fullsized":"2024-04-26T20:22:41.762202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:19.607106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751655+0000","last_change":"2024-04-26T20:22:41.751655+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751655+0000","last_undegraded":"2024-04-26T20:22:41.751655+0000","last_fullsized":"2024-04-26T20:22:41.751655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761347+0000","last_change":"2024-04-26T20:22:41.760401+0000","last_active":"2024-04-26T20:22:41.761347+0000","last_peered":"2024-04-26T20:22:41.761347+0000","last_clean":"2024-04-26T20:22:41.761347+0000","last_became_active":"2024-04-26T20:22:41.760080+0000","last_became_peered":"2024-04-26T20:22:41.760080+0000","last_unstale":"2024-04-26T20:22:41.761347+0000","last_undegraded":"2024-04-26T20:22:41.761347+0000","last_fullsized":"2024-04-26T20:22:41.761347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:02.047370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744516+0000","last_change":"2024-04-26T20:22:41.744516+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744516+0000","last_undegraded":"2024-04-26T20:22:41.744516+0000","last_fullsized":"2024-04-26T20:22:41.744516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760372+0000","last_change":"2024-04-26T20:22:41.756976+0000","last_active":"2024-04-26T20:22:41.760372+0000","last_peered":"2024-04-26T20:22:41.760372+0000","last_clean":"2024-04-26T20:22:41.760372+0000","last_became_active":"2024-04-26T20:22:41.756639+0000","last_became_peered":"2024-04-26T20:22:41.756639+0000","last_unstale":"2024-04-26T20:22:41.760372+0000","last_undegraded":"2024-04-26T20:22:41.760372+0000","last_fullsized":"2024-04-26T20:22:41.760372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751992+0000","last_change":"2024-04-26T20:22:41.751992+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751992+0000","last_undegraded":"2024-04-26T20:22:41.751992+0000","last_fullsized":"2024-04-26T20:22:41.751992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.749878+0000","last_change":"2024-04-26T20:22:41.747756+0000","last_active":"2024-04-26T20:22:41.749878+0000","last_peered":"2024-04-26T20:22:41.749878+0000","last_clean":"2024-04-26T20:22:41.749878+0000","last_became_active":"2024-04-26T20:22:41.747460+0000","last_became_peered":"2024-04-26T20:22:41.747460+0000","last_unstale":"2024-04-26T20:22:41.749878+0000","last_undegraded":"2024-04-26T20:22:41.749878+0000","last_fullsized":"2024-04-26T20:22:41.749878+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.742836+0000","last_change":"2024-04-26T20:22:41.742836+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.742836+0000","last_undegraded":"2024-04-26T20:22:41.742836+0000","last_fullsized":"2024-04-26T20:22:41.742836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758269+0000","last_change":"2024-04-26T20:22:41.750670+0000","last_active":"2024-04-26T20:22:41.758269+0000","last_peered":"2024-04-26T20:22:41.758269+0000","last_clean":"2024-04-26T20:22:41.758269+0000","last_became_active":"2024-04-26T20:22:41.750317+0000","last_became_peered":"2024-04-26T20:22:41.750317+0000","last_unstale":"2024-04-26T20:22:41.758269+0000","last_undegraded":"2024-04-26T20:22:41.758269+0000","last_fullsized":"2024-04-26T20:22:41.758269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:55.932492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.747088+0000","last_change":"2024-04-26T20:22:41.747088+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.747088+0000","last_undegraded":"2024-04-26T20:22:41.747088+0000","last_fullsized":"2024-04-26T20:22:41.747088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.756385+0000","last_change":"2024-04-26T20:22:41.754948+0000","last_active":"2024-04-26T20:22:41.756385+0000","last_peered":"2024-04-26T20:22:41.756385+0000","last_clean":"2024-04-26T20:22:41.756385+0000","last_became_active":"2024-04-26T20:22:41.754664+0000","last_became_peered":"2024-04-26T20:22:41.754664+0000","last_unstale":"2024-04-26T20:22:41.756385+0000","last_undegraded":"2024-04-26T20:22:41.756385+0000","last_fullsized":"2024-04-26T20:22:41.756385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743378+0000","last_change":"2024-04-26T20:22:41.743378+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743378+0000","last_undegraded":"2024-04-26T20:22:41.743378+0000","last_fullsized":"2024-04-26T20:22:41.743378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760503+0000","last_change":"2024-04-26T20:22:41.756879+0000","last_active":"2024-04-26T20:22:41.760503+0000","last_peered":"2024-04-26T20:22:41.760503+0000","last_clean":"2024-04-26T20:22:41.760503+0000","last_became_active":"2024-04-26T20:22:41.756600+0000","last_became_peered":"2024-04-26T20:22:41.756600+0000","last_unstale":"2024-04-26T20:22:41.760503+0000","last_undegraded":"2024-04-26T20:22:41.760503+0000","last_fullsized":"2024-04-26T20:22:41.760503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750965+0000","last_change":"2024-04-26T20:22:41.750965+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750965+0000","last_undegraded":"2024-04-26T20:22:41.750965+0000","last_fullsized":"2024-04-26T20:22:41.750965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.759460+0000","last_change":"2024-04-26T20:22:41.756827+0000","last_active":"2024-04-26T20:22:41.759460+0000","last_peered":"2024-04-26T20:22:41.759460+0000","last_clean":"2024-04-26T20:22:41.759460+0000","last_became_active":"2024-04-26T20:22:41.756542+0000","last_became_peered":"2024-04-26T20:22:41.756542+0000","last_unstale":"2024-04-26T20:22:41.759460+0000","last_undegraded":"2024-04-26T20:22:41.759460+0000","last_fullsized":"2024-04-26T20:22:41.759460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:52.253630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748484+0000","last_change":"2024-04-26T20:22:41.748484+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748484+0000","last_undegraded":"2024-04-26T20:22:41.748484+0000","last_fullsized":"2024-04-26T20:22:41.748484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765234+0000","last_change":"2024-04-26T20:22:41.760981+0000","last_active":"2024-04-26T20:22:41.765234+0000","last_peered":"2024-04-26T20:22:41.765234+0000","last_clean":"2024-04-26T20:22:41.765234+0000","last_became_active":"2024-04-26T20:22:41.760700+0000","last_became_peered":"2024-04-26T20:22:41.760700+0000","last_unstale":"2024-04-26T20:22:41.765234+0000","last_undegraded":"2024-04-26T20:22:41.765234+0000","last_fullsized":"2024-04-26T20:22:41.765234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:20.612907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750327+0000","last_change":"2024-04-26T20:22:41.750327+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750327+0000","last_undegraded":"2024-04-26T20:22:41.750327+0000","last_fullsized":"2024-04-26T20:22:41.750327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760640+0000","last_change":"2024-04-26T20:22:41.755373+0000","last_active":"2024-04-26T20:22:41.760640+0000","last_peered":"2024-04-26T20:22:41.760640+0000","last_clean":"2024-04-26T20:22:41.760640+0000","last_became_active":"2024-04-26T20:22:41.755087+0000","last_became_peered":"2024-04-26T20:22:41.755087+0000","last_unstale":"2024-04-26T20:22:41.760640+0000","last_undegraded":"2024-04-26T20:22:41.760640+0000","last_fullsized":"2024-04-26T20:22:41.760640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.756741+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:22:41.756741+0000","last_peered":"2024-04-26T20:22:41.756741+0000","last_clean":"2024-04-26T20:22:41.756741+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:22:41.756741+0000","last_undegraded":"2024-04-26T20:22:41.756741+0000","last_fullsized":"2024-04-26T20:22:41.756741+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:22:47.127 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:22:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.262+0000 7f945bf67640 1 Processor -- start 2024-04-26T20:22:47.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.263+0000 7f945bf67640 1 -- start start 2024-04-26T20:22:47.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.263+0000 7f945bf67640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f945406c240 0x7f945406fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.263+0000 7f945bf67640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f945415f4f0 0x7f945406c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.263+0000 7f945bf67640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f945406cb60 con 0x7f94540b1480 2024-04-26T20:22:47.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.263+0000 7f945bf67640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f94540b59b0 con 0x7f945406c240 2024-04-26T20:22:47.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.263+0000 7f945bf67640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9454070350 con 0x7f945415f4f0 2024-04-26T20:22:47.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.264+0000 7f9459d03640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f945406c240 0x7f945406fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.264+0000 7f945a504640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f94540b1480 0x7f94540ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52578/0 (socket says 172.21.15.37:52578) 2024-04-26T20:22:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.264+0000 7f945a504640 1 -- 172.21.15.37:0/1418458110 learned_addr learned my addr 172.21.15.37:0/1418458110 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.264+0000 7f9459502640 1 --2- 172.21.15.37:0/1418458110 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f945415f4f0 0x7f945406c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.264+0000 7f9459d03640 1 -- 172.21.15.37:0/1418458110 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f945415f4f0 msgr2=0x7f945406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.264+0000 7f9459d03640 1 --2- 172.21.15.37:0/1418458110 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f945415f4f0 0x7f945406c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.264+0000 7f9459d03640 1 -- 172.21.15.37:0/1418458110 >> v1:172.21.15.37:6789/0 conn(0x7f94540b1480 legacy=0x7f94540ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.264+0000 7f9458d01640 1 -- 172.21.15.37:0/1418458110 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3617226155 0 0) 0x7f945406cb60 con 0x7f94540b1480 2024-04-26T20:22:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.264+0000 7f9459d03640 1 -- 172.21.15.37:0/1418458110 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94540b5c70 con 0x7f945406c240 2024-04-26T20:22:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.265+0000 7f9459d03640 1 --2- 172.21.15.37:0/1418458110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f945406c240 0x7f945406fd70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f94500097a0 tx=0x7f945002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30670afab89b9c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.265+0000 7f9458d01640 1 -- 172.21.15.37:0/1418458110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450037070 con 0x7f945406c240 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.265+0000 7f9458d01640 1 -- 172.21.15.37:0/1418458110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9450032440 con 0x7f945406c240 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.265+0000 7f9458d01640 1 -- 172.21.15.37:0/1418458110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450032810 con 0x7f945406c240 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.265+0000 7f945bf67640 1 -- 172.21.15.37:0/1418458110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f945406c240 msgr2=0x7f945406fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.265+0000 7f945bf67640 1 --2- 172.21.15.37:0/1418458110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f945406c240 0x7f945406fd70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f94500097a0 tx=0x7f945002a180 comp rx=0 tx=0).stop 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.265+0000 7f945bf67640 1 -- 172.21.15.37:0/1418458110 shutdown_connections 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.265+0000 7f945bf67640 1 --2- 172.21.15.37:0/1418458110 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f945415f4f0 0x7f945406c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.265+0000 7f945bf67640 1 --2- 172.21.15.37:0/1418458110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f945406c240 0x7f945406fd70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.265+0000 7f945bf67640 1 -- 172.21.15.37:0/1418458110 >> 172.21.15.37:0/1418458110 conn(0x7f94540b3b70 msgr2=0x7f94540b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.266+0000 7f945bf67640 1 -- 172.21.15.37:0/1418458110 shutdown_connections 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.266+0000 7f945bf67640 1 -- 172.21.15.37:0/1418458110 wait complete. 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.267+0000 7f945bf67640 1 Processor -- start 2024-04-26T20:22:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.267+0000 7f945bf67640 1 -- start start 2024-04-26T20:22:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.267+0000 7f945bf67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f945406c240 0x7f9454151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.267+0000 7f945bf67640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94540b1480 0x7f9454155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.267+0000 7f945bf67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f945415f4f0 0x7f9454154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.267+0000 7f945bf67640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f94540b0fc0 con 0x7f945415f4f0 2024-04-26T20:22:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.267+0000 7f945bf67640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94540b6230 con 0x7f945406c240 2024-04-26T20:22:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.267+0000 7f945bf67640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f94540b6750 con 0x7f94540b1480 2024-04-26T20:22:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f945a504640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f945415f4f0 0x7f9454154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f945a504640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f945415f4f0 0x7f9454154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35850/0 (socket says 172.21.15.37:35850) 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f945a504640 1 -- 172.21.15.37:0/3684002274 learned_addr learned my addr 172.21.15.37:0/3684002274 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f9459502640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94540b1480 0x7f9454155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f9459d03640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f945406c240 0x7f9454151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f945a504640 1 -- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94540b1480 msgr2=0x7f9454155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f945a504640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94540b1480 0x7f9454155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f945a504640 1 -- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f945406c240 msgr2=0x7f9454151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f945a504640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f945406c240 0x7f9454151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f945a504640 1 -- 172.21.15.37:0/3684002274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9454154640 con 0x7f945415f4f0 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f945a504640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f945415f4f0 0x7f9454154cf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c7a0 tx=0x7f944c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f944a7fc640 1 -- 172.21.15.37:0/3684002274 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c018070 con 0x7f945415f4f0 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f944a7fc640 1 -- 172.21.15.37:0/3684002274 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944c00a640 con 0x7f945415f4f0 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f945bf67640 1 -- 172.21.15.37:0/3684002274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94541614c0 con 0x7f945415f4f0 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f944a7fc640 1 -- 172.21.15.37:0/3684002274 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c00aab0 con 0x7f945415f4f0 2024-04-26T20:22:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.268+0000 7f945bf67640 1 -- 172.21.15.37:0/3684002274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9454161760 con 0x7f945415f4f0 2024-04-26T20:22:47.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.269+0000 7f945bf67640 1 -- 172.21.15.37:0/3684002274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94540b09b0 con 0x7f945415f4f0 2024-04-26T20:22:47.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.270+0000 7f944a7fc640 1 -- 172.21.15.37:0/3684002274 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f944c01b070 con 0x7f945415f4f0 2024-04-26T20:22:47.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.270+0000 7f944a7fc640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f943806c8f0 0x7f943806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.270+0000 7f944a7fc640 1 -- 172.21.15.37:0/3684002274 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f944c08b090 con 0x7f945415f4f0 2024-04-26T20:22:47.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.270+0000 7f9459d03640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f943806c8f0 0x7f943806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.271+0000 7f9459d03640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f943806c8f0 0x7f943806edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9450034000 tx=0x7f945002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:47.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.274+0000 7f944a7fc640 1 -- 172.21.15.37:0/3684002274 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94540b09b0 con 0x7f945415f4f0 2024-04-26T20:22:47.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.458+0000 7f945bf67640 1 -- 172.21.15.37:0/3684002274 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f945406a360 con 0x7f943806c8f0 2024-04-26T20:22:47.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.465+0000 7f944a7fc640 1 -- 172.21.15.37:0/3684002274 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7f945406a360 con 0x7f943806c8f0 2024-04-26T20:22:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:47.469 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:22:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 -- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f943806c8f0 msgr2=0x7f943806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f943806c8f0 0x7f943806edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9450034000 tx=0x7f945002e000 comp rx=0 tx=0).stop 2024-04-26T20:22:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 -- 172.21.15.37:0/3684002274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f945415f4f0 msgr2=0x7f9454154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f945415f4f0 0x7f9454154cf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c7a0 tx=0x7f944c00cc60 comp rx=0 tx=0).stop 2024-04-26T20:22:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 -- 172.21.15.37:0/3684002274 shutdown_connections 2024-04-26T20:22:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f945415f4f0 0x7f9454154cf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f943806c8f0 0x7f943806edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94540b1480 0x7f9454155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 --2- 172.21.15.37:0/3684002274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f945406c240 0x7f9454151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 -- 172.21.15.37:0/3684002274 >> 172.21.15.37:0/3684002274 conn(0x7f94540b3b70 msgr2=0x7f94540b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:47.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 -- 172.21.15.37:0/3684002274 shutdown_connections 2024-04-26T20:22:47.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.468+0000 7f945bf67640 1 -- 172.21.15.37:0/3684002274 wait complete. 2024-04-26T20:22:47.479 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-04-26T20:22:47.347078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220008,"kb_used_data":3832,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754712,"statfs":{"total":773094113280,"available":772868825088,"internally_reserved":0,"allocated":3923968,"data_stored":2223996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001014"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.747315+0000","last_change":"2024-04-26T20:22:41.747315+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.747315+0000","last_undegraded":"2024-04-26T20:22:41.747315+0000","last_fullsized":"2024-04-26T20:22:41.747315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751176+0000","last_change":"2024-04-26T20:22:41.751176+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751176+0000","last_undegraded":"2024-04-26T20:22:41.751176+0000","last_fullsized":"2024-04-26T20:22:41.751176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749201+0000","last_change":"2024-04-26T20:22:41.749201+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749201+0000","last_undegraded":"2024-04-26T20:22:41.749201+0000","last_fullsized":"2024-04-26T20:22:41.749201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745545+0000","last_change":"2024-04-26T20:22:41.745545+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745545+0000","last_undegraded":"2024-04-26T20:22:41.745545+0000","last_fullsized":"2024-04-26T20:22:41.745545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745404+0000","last_change":"2024-04-26T20:22:41.745404+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745404+0000","last_undegraded":"2024-04-26T20:22:41.745404+0000","last_fullsized":"2024-04-26T20:22:41.745404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750104+0000","last_change":"2024-04-26T20:22:41.750104+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750104+0000","last_undegraded":"2024-04-26T20:22:41.750104+0000","last_fullsized":"2024-04-26T20:22:41.750104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750969+0000","last_change":"2024-04-26T20:22:41.750969+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750969+0000","last_undegraded":"2024-04-26T20:22:41.750969+0000","last_fullsized":"2024-04-26T20:22:41.750969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753949+0000","last_change":"2024-04-26T20:22:41.753949+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753949+0000","last_undegraded":"2024-04-26T20:22:41.753949+0000","last_fullsized":"2024-04-26T20:22:41.753949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749215+0000","last_change":"2024-04-26T20:22:41.749215+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749215+0000","last_undegraded":"2024-04-26T20:22:41.749215+0000","last_fullsized":"2024-04-26T20:22:41.749215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744734+0000","last_change":"2024-04-26T20:22:41.744734+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744734+0000","last_undegraded":"2024-04-26T20:22:41.744734+0000","last_fullsized":"2024-04-26T20:22:41.744734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751008+0000","last_change":"2024-04-26T20:22:41.751008+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751008+0000","last_undegraded":"2024-04-26T20:22:41.751008+0000","last_fullsized":"2024-04-26T20:22:41.751008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751513+0000","last_change":"2024-04-26T20:22:41.751513+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751513+0000","last_undegraded":"2024-04-26T20:22:41.751513+0000","last_fullsized":"2024-04-26T20:22:41.751513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744914+0000","last_change":"2024-04-26T20:22:41.744914+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744914+0000","last_undegraded":"2024-04-26T20:22:41.744914+0000","last_fullsized":"2024-04-26T20:22:41.744914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749030+0000","last_change":"2024-04-26T20:22:41.749030+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749030+0000","last_undegraded":"2024-04-26T20:22:41.749030+0000","last_fullsized":"2024-04-26T20:22:41.749030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750437+0000","last_change":"2024-04-26T20:22:41.750437+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750437+0000","last_undegraded":"2024-04-26T20:22:41.750437+0000","last_fullsized":"2024-04-26T20:22:41.750437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751270+0000","last_change":"2024-04-26T20:22:41.751270+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751270+0000","last_undegraded":"2024-04-26T20:22:41.751270+0000","last_fullsized":"2024-04-26T20:22:41.751270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751848+0000","last_change":"2024-04-26T20:22:41.751848+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751848+0000","last_undegraded":"2024-04-26T20:22:41.751848+0000","last_fullsized":"2024-04-26T20:22:41.751848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750736+0000","last_change":"2024-04-26T20:22:41.750736+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750736+0000","last_undegraded":"2024-04-26T20:22:41.750736+0000","last_fullsized":"2024-04-26T20:22:41.750736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750525+0000","last_change":"2024-04-26T20:22:41.750525+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750525+0000","last_undegraded":"2024-04-26T20:22:41.750525+0000","last_fullsized":"2024-04-26T20:22:41.750525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744169+0000","last_change":"2024-04-26T20:22:41.744169+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744169+0000","last_undegraded":"2024-04-26T20:22:41.744169+0000","last_fullsized":"2024-04-26T20:22:41.744169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748990+0000","last_change":"2024-04-26T20:22:41.748990+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748990+0000","last_undegraded":"2024-04-26T20:22:41.748990+0000","last_fullsized":"2024-04-26T20:22:41.748990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750071+0000","last_change":"2024-04-26T20:22:41.750071+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750071+0000","last_undegraded":"2024-04-26T20:22:41.750071+0000","last_fullsized":"2024-04-26T20:22:41.750071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744336+0000","last_change":"2024-04-26T20:22:41.744336+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744336+0000","last_undegraded":"2024-04-26T20:22:41.744336+0000","last_fullsized":"2024-04-26T20:22:41.744336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749073+0000","last_change":"2024-04-26T20:22:41.749073+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749073+0000","last_undegraded":"2024-04-26T20:22:41.749073+0000","last_fullsized":"2024-04-26T20:22:41.749073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745963+0000","last_change":"2024-04-26T20:22:41.745963+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745963+0000","last_undegraded":"2024-04-26T20:22:41.745963+0000","last_fullsized":"2024-04-26T20:22:41.745963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750267+0000","last_change":"2024-04-26T20:22:41.750267+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750267+0000","last_undegraded":"2024-04-26T20:22:41.750267+0000","last_fullsized":"2024-04-26T20:22:41.750267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744351+0000","last_change":"2024-04-26T20:22:41.744351+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744351+0000","last_undegraded":"2024-04-26T20:22:41.744351+0000","last_fullsized":"2024-04-26T20:22:41.744351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748823+0000","last_change":"2024-04-26T20:22:41.748823+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748823+0000","last_undegraded":"2024-04-26T20:22:41.748823+0000","last_fullsized":"2024-04-26T20:22:41.748823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748525+0000","last_change":"2024-04-26T20:22:41.748525+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748525+0000","last_undegraded":"2024-04-26T20:22:41.748525+0000","last_fullsized":"2024-04-26T20:22:41.748525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.752597+0000","last_change":"2024-04-26T20:22:41.752597+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.752597+0000","last_undegraded":"2024-04-26T20:22:41.752597+0000","last_fullsized":"2024-04-26T20:22:41.752597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.746522+0000","last_change":"2024-04-26T20:22:41.746522+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.746522+0000","last_undegraded":"2024-04-26T20:22:41.746522+0000","last_fullsized":"2024-04-26T20:22:41.746522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753519+0000","last_change":"2024-04-26T20:22:41.753519+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753519+0000","last_undegraded":"2024-04-26T20:22:41.753519+0000","last_fullsized":"2024-04-26T20:22:41.753519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744224+0000","last_change":"2024-04-26T20:22:41.744224+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744224+0000","last_undegraded":"2024-04-26T20:22:41.744224+0000","last_fullsized":"2024-04-26T20:22:41.744224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760557+0000","last_change":"2024-04-26T20:22:41.756667+0000","last_active":"2024-04-26T20:22:41.760557+0000","last_peered":"2024-04-26T20:22:41.760557+0000","last_clean":"2024-04-26T20:22:41.760557+0000","last_became_active":"2024-04-26T20:22:41.756385+0000","last_became_peered":"2024-04-26T20:22:41.756385+0000","last_unstale":"2024-04-26T20:22:41.760557+0000","last_undegraded":"2024-04-26T20:22:41.760557+0000","last_fullsized":"2024-04-26T20:22:41.760557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.746845+0000","last_change":"2024-04-26T20:22:41.746845+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.746845+0000","last_undegraded":"2024-04-26T20:22:41.746845+0000","last_fullsized":"2024-04-26T20:22:41.746845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764642+0000","last_change":"2024-04-26T20:22:41.758415+0000","last_active":"2024-04-26T20:22:41.764642+0000","last_peered":"2024-04-26T20:22:41.764642+0000","last_clean":"2024-04-26T20:22:41.764642+0000","last_became_active":"2024-04-26T20:22:41.757712+0000","last_became_peered":"2024-04-26T20:22:41.757712+0000","last_unstale":"2024-04-26T20:22:41.764642+0000","last_undegraded":"2024-04-26T20:22:41.764642+0000","last_fullsized":"2024-04-26T20:22:41.764642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:38.972810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749597+0000","last_change":"2024-04-26T20:22:41.749597+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749597+0000","last_undegraded":"2024-04-26T20:22:41.749597+0000","last_fullsized":"2024-04-26T20:22:41.749597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764752+0000","last_change":"2024-04-26T20:22:41.759486+0000","last_active":"2024-04-26T20:22:41.764752+0000","last_peered":"2024-04-26T20:22:41.764752+0000","last_clean":"2024-04-26T20:22:41.764752+0000","last_became_active":"2024-04-26T20:22:41.759076+0000","last_became_peered":"2024-04-26T20:22:41.759076+0000","last_unstale":"2024-04-26T20:22:41.764752+0000","last_undegraded":"2024-04-26T20:22:41.764752+0000","last_fullsized":"2024-04-26T20:22:41.764752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:27.337131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743760+0000","last_change":"2024-04-26T20:22:41.743760+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743760+0000","last_undegraded":"2024-04-26T20:22:41.743760+0000","last_fullsized":"2024-04-26T20:22:41.743760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758241+0000","last_change":"2024-04-26T20:22:41.754766+0000","last_active":"2024-04-26T20:22:41.758241+0000","last_peered":"2024-04-26T20:22:41.758241+0000","last_clean":"2024-04-26T20:22:41.758241+0000","last_became_active":"2024-04-26T20:22:41.754427+0000","last_became_peered":"2024-04-26T20:22:41.754427+0000","last_unstale":"2024-04-26T20:22:41.758241+0000","last_undegraded":"2024-04-26T20:22:41.758241+0000","last_fullsized":"2024-04-26T20:22:41.758241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:52.253630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750686+0000","last_change":"2024-04-26T20:22:41.750686+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750686+0000","last_undegraded":"2024-04-26T20:22:41.750686+0000","last_fullsized":"2024-04-26T20:22:41.750686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762211+0000","last_change":"2024-04-26T20:22:41.759951+0000","last_active":"2024-04-26T20:22:41.762211+0000","last_peered":"2024-04-26T20:22:41.762211+0000","last_clean":"2024-04-26T20:22:41.762211+0000","last_became_active":"2024-04-26T20:22:41.759681+0000","last_became_peered":"2024-04-26T20:22:41.759681+0000","last_unstale":"2024-04-26T20:22:41.762211+0000","last_undegraded":"2024-04-26T20:22:41.762211+0000","last_fullsized":"2024-04-26T20:22:41.762211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:49.082266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750653+0000","last_change":"2024-04-26T20:22:41.750653+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750653+0000","last_undegraded":"2024-04-26T20:22:41.750653+0000","last_fullsized":"2024-04-26T20:22:41.750653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761855+0000","last_change":"2024-04-26T20:22:41.758461+0000","last_active":"2024-04-26T20:22:41.761855+0000","last_peered":"2024-04-26T20:22:41.761855+0000","last_clean":"2024-04-26T20:22:41.761855+0000","last_became_active":"2024-04-26T20:22:41.758166+0000","last_became_peered":"2024-04-26T20:22:41.758166+0000","last_unstale":"2024-04-26T20:22:41.761855+0000","last_undegraded":"2024-04-26T20:22:41.761855+0000","last_fullsized":"2024-04-26T20:22:41.761855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:24.346362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749681+0000","last_change":"2024-04-26T20:22:41.749681+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749681+0000","last_undegraded":"2024-04-26T20:22:41.749681+0000","last_fullsized":"2024-04-26T20:22:41.749681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760841+0000","last_change":"2024-04-26T20:22:41.754578+0000","last_active":"2024-04-26T20:22:41.760841+0000","last_peered":"2024-04-26T20:22:41.760841+0000","last_clean":"2024-04-26T20:22:41.760841+0000","last_became_active":"2024-04-26T20:22:41.754286+0000","last_became_peered":"2024-04-26T20:22:41.754286+0000","last_unstale":"2024-04-26T20:22:41.760841+0000","last_undegraded":"2024-04-26T20:22:41.760841+0000","last_fullsized":"2024-04-26T20:22:41.760841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743070+0000","last_change":"2024-04-26T20:22:41.743070+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743070+0000","last_undegraded":"2024-04-26T20:22:41.743070+0000","last_fullsized":"2024-04-26T20:22:41.743070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765354+0000","last_change":"2024-04-26T20:22:41.762053+0000","last_active":"2024-04-26T20:22:41.765354+0000","last_peered":"2024-04-26T20:22:41.765354+0000","last_clean":"2024-04-26T20:22:41.765354+0000","last_became_active":"2024-04-26T20:22:41.761753+0000","last_became_peered":"2024-04-26T20:22:41.761753+0000","last_unstale":"2024-04-26T20:22:41.765354+0000","last_undegraded":"2024-04-26T20:22:41.765354+0000","last_fullsized":"2024-04-26T20:22:41.765354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:36.559062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748362+0000","last_change":"2024-04-26T20:22:41.748362+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748362+0000","last_undegraded":"2024-04-26T20:22:41.748362+0000","last_fullsized":"2024-04-26T20:22:41.748362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765262+0000","last_change":"2024-04-26T20:22:41.761862+0000","last_active":"2024-04-26T20:22:41.765262+0000","last_peered":"2024-04-26T20:22:41.765262+0000","last_clean":"2024-04-26T20:22:41.765262+0000","last_became_active":"2024-04-26T20:22:41.761611+0000","last_became_peered":"2024-04-26T20:22:41.761611+0000","last_unstale":"2024-04-26T20:22:41.765262+0000","last_undegraded":"2024-04-26T20:22:41.765262+0000","last_fullsized":"2024-04-26T20:22:41.765262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:02.047370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749794+0000","last_change":"2024-04-26T20:22:41.749794+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749794+0000","last_undegraded":"2024-04-26T20:22:41.749794+0000","last_fullsized":"2024-04-26T20:22:41.749794+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764613+0000","last_change":"2024-04-26T20:22:41.757671+0000","last_active":"2024-04-26T20:22:41.764613+0000","last_peered":"2024-04-26T20:22:41.764613+0000","last_clean":"2024-04-26T20:22:41.764613+0000","last_became_active":"2024-04-26T20:22:41.757277+0000","last_became_peered":"2024-04-26T20:22:41.757277+0000","last_unstale":"2024-04-26T20:22:41.764613+0000","last_undegraded":"2024-04-26T20:22:41.764613+0000","last_fullsized":"2024-04-26T20:22:41.764613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749475+0000","last_change":"2024-04-26T20:22:41.749475+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749475+0000","last_undegraded":"2024-04-26T20:22:41.749475+0000","last_fullsized":"2024-04-26T20:22:41.749475+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762294+0000","last_change":"2024-04-26T20:22:41.760284+0000","last_active":"2024-04-26T20:22:41.762294+0000","last_peered":"2024-04-26T20:22:41.762294+0000","last_clean":"2024-04-26T20:22:41.762294+0000","last_became_active":"2024-04-26T20:22:41.759981+0000","last_became_peered":"2024-04-26T20:22:41.759981+0000","last_unstale":"2024-04-26T20:22:41.762294+0000","last_undegraded":"2024-04-26T20:22:41.762294+0000","last_fullsized":"2024-04-26T20:22:41.762294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:20.612907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749667+0000","last_change":"2024-04-26T20:22:41.749667+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749667+0000","last_undegraded":"2024-04-26T20:22:41.749667+0000","last_fullsized":"2024-04-26T20:22:41.749667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764722+0000","last_change":"2024-04-26T20:22:41.759409+0000","last_active":"2024-04-26T20:22:41.764722+0000","last_peered":"2024-04-26T20:22:41.764722+0000","last_clean":"2024-04-26T20:22:41.764722+0000","last_became_active":"2024-04-26T20:22:41.758974+0000","last_became_peered":"2024-04-26T20:22:41.758974+0000","last_unstale":"2024-04-26T20:22:41.764722+0000","last_undegraded":"2024-04-26T20:22:41.764722+0000","last_fullsized":"2024-04-26T20:22:41.764722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:38.972810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743856+0000","last_change":"2024-04-26T20:22:41.743856+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743856+0000","last_undegraded":"2024-04-26T20:22:41.743856+0000","last_fullsized":"2024-04-26T20:22:41.743856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762234+0000","last_change":"2024-04-26T20:22:41.760955+0000","last_active":"2024-04-26T20:22:41.762234+0000","last_peered":"2024-04-26T20:22:41.762234+0000","last_clean":"2024-04-26T20:22:41.762234+0000","last_became_active":"2024-04-26T20:22:41.760658+0000","last_became_peered":"2024-04-26T20:22:41.760658+0000","last_unstale":"2024-04-26T20:22:41.762234+0000","last_undegraded":"2024-04-26T20:22:41.762234+0000","last_fullsized":"2024-04-26T20:22:41.762234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:42.223536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748548+0000","last_change":"2024-04-26T20:22:41.748548+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748548+0000","last_undegraded":"2024-04-26T20:22:41.748548+0000","last_fullsized":"2024-04-26T20:22:41.748548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758419+0000","last_change":"2024-04-26T20:22:41.751412+0000","last_active":"2024-04-26T20:22:41.758419+0000","last_peered":"2024-04-26T20:22:41.758419+0000","last_clean":"2024-04-26T20:22:41.758419+0000","last_became_active":"2024-04-26T20:22:41.751102+0000","last_became_peered":"2024-04-26T20:22:41.751102+0000","last_unstale":"2024-04-26T20:22:41.758419+0000","last_undegraded":"2024-04-26T20:22:41.758419+0000","last_fullsized":"2024-04-26T20:22:41.758419+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753116+0000","last_change":"2024-04-26T20:22:41.753116+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753116+0000","last_undegraded":"2024-04-26T20:22:41.753116+0000","last_fullsized":"2024-04-26T20:22:41.753116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765217+0000","last_change":"2024-04-26T20:22:41.760709+0000","last_active":"2024-04-26T20:22:41.765217+0000","last_peered":"2024-04-26T20:22:41.765217+0000","last_clean":"2024-04-26T20:22:41.765217+0000","last_became_active":"2024-04-26T20:22:41.760281+0000","last_became_peered":"2024-04-26T20:22:41.760281+0000","last_unstale":"2024-04-26T20:22:41.765217+0000","last_undegraded":"2024-04-26T20:22:41.765217+0000","last_fullsized":"2024-04-26T20:22:41.765217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:20.522370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745965+0000","last_change":"2024-04-26T20:22:41.745965+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745965+0000","last_undegraded":"2024-04-26T20:22:41.745965+0000","last_fullsized":"2024-04-26T20:22:41.745965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764753+0000","last_change":"2024-04-26T20:22:41.759620+0000","last_active":"2024-04-26T20:22:41.764753+0000","last_peered":"2024-04-26T20:22:41.764753+0000","last_clean":"2024-04-26T20:22:41.764753+0000","last_became_active":"2024-04-26T20:22:41.759154+0000","last_became_peered":"2024-04-26T20:22:41.759154+0000","last_unstale":"2024-04-26T20:22:41.764753+0000","last_undegraded":"2024-04-26T20:22:41.764753+0000","last_fullsized":"2024-04-26T20:22:41.764753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:42.413094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750914+0000","last_change":"2024-04-26T20:22:41.750914+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750914+0000","last_undegraded":"2024-04-26T20:22:41.750914+0000","last_fullsized":"2024-04-26T20:22:41.750914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761309+0000","last_change":"2024-04-26T20:22:41.758134+0000","last_active":"2024-04-26T20:22:41.761309+0000","last_peered":"2024-04-26T20:22:41.761309+0000","last_clean":"2024-04-26T20:22:41.761309+0000","last_became_active":"2024-04-26T20:22:41.757853+0000","last_became_peered":"2024-04-26T20:22:41.757853+0000","last_unstale":"2024-04-26T20:22:41.761309+0000","last_undegraded":"2024-04-26T20:22:41.761309+0000","last_fullsized":"2024-04-26T20:22:41.761309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.752554+0000","last_change":"2024-04-26T20:22:41.752554+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.752554+0000","last_undegraded":"2024-04-26T20:22:41.752554+0000","last_fullsized":"2024-04-26T20:22:41.752554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758900+0000","last_change":"2024-04-26T20:22:41.756567+0000","last_active":"2024-04-26T20:22:41.758900+0000","last_peered":"2024-04-26T20:22:41.758900+0000","last_clean":"2024-04-26T20:22:41.758900+0000","last_became_active":"2024-04-26T20:22:41.756259+0000","last_became_peered":"2024-04-26T20:22:41.756259+0000","last_unstale":"2024-04-26T20:22:41.758900+0000","last_undegraded":"2024-04-26T20:22:41.758900+0000","last_fullsized":"2024-04-26T20:22:41.758900+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:19.518020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745137+0000","last_change":"2024-04-26T20:22:41.745137+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745137+0000","last_undegraded":"2024-04-26T20:22:41.745137+0000","last_fullsized":"2024-04-26T20:22:41.745137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761886+0000","last_change":"2024-04-26T20:22:41.757636+0000","last_active":"2024-04-26T20:22:41.761886+0000","last_peered":"2024-04-26T20:22:41.761886+0000","last_clean":"2024-04-26T20:22:41.761886+0000","last_became_active":"2024-04-26T20:22:41.757329+0000","last_became_peered":"2024-04-26T20:22:41.757329+0000","last_unstale":"2024-04-26T20:22:41.761886+0000","last_undegraded":"2024-04-26T20:22:41.761886+0000","last_fullsized":"2024-04-26T20:22:41.761886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:38.757083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744642+0000","last_change":"2024-04-26T20:22:41.744642+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744642+0000","last_undegraded":"2024-04-26T20:22:41.744642+0000","last_fullsized":"2024-04-26T20:22:41.744642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762077+0000","last_change":"2024-04-26T20:22:41.759343+0000","last_active":"2024-04-26T20:22:41.762077+0000","last_peered":"2024-04-26T20:22:41.762077+0000","last_clean":"2024-04-26T20:22:41.762077+0000","last_became_active":"2024-04-26T20:22:41.758924+0000","last_became_peered":"2024-04-26T20:22:41.758924+0000","last_unstale":"2024-04-26T20:22:41.762077+0000","last_undegraded":"2024-04-26T20:22:41.762077+0000","last_fullsized":"2024-04-26T20:22:41.762077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.731025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743229+0000","last_change":"2024-04-26T20:22:41.743229+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743229+0000","last_undegraded":"2024-04-26T20:22:41.743229+0000","last_fullsized":"2024-04-26T20:22:41.743229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.749913+0000","last_change":"2024-04-26T20:22:41.747174+0000","last_active":"2024-04-26T20:22:41.749913+0000","last_peered":"2024-04-26T20:22:41.749913+0000","last_clean":"2024-04-26T20:22:41.749913+0000","last_became_active":"2024-04-26T20:22:41.746153+0000","last_became_peered":"2024-04-26T20:22:41.746153+0000","last_unstale":"2024-04-26T20:22:41.749913+0000","last_undegraded":"2024-04-26T20:22:41.749913+0000","last_fullsized":"2024-04-26T20:22:41.749913+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743176+0000","last_change":"2024-04-26T20:22:41.743176+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743176+0000","last_undegraded":"2024-04-26T20:22:41.743176+0000","last_fullsized":"2024-04-26T20:22:41.743176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760527+0000","last_change":"2024-04-26T20:22:41.755144+0000","last_active":"2024-04-26T20:22:41.760527+0000","last_peered":"2024-04-26T20:22:41.760527+0000","last_clean":"2024-04-26T20:22:41.760527+0000","last_became_active":"2024-04-26T20:22:41.754847+0000","last_became_peered":"2024-04-26T20:22:41.754847+0000","last_unstale":"2024-04-26T20:22:41.760527+0000","last_undegraded":"2024-04-26T20:22:41.760527+0000","last_fullsized":"2024-04-26T20:22:41.760527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745108+0000","last_change":"2024-04-26T20:22:41.745108+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745108+0000","last_undegraded":"2024-04-26T20:22:41.745108+0000","last_fullsized":"2024-04-26T20:22:41.745108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762202+0000","last_change":"2024-04-26T20:22:41.759321+0000","last_active":"2024-04-26T20:22:41.762202+0000","last_peered":"2024-04-26T20:22:41.762202+0000","last_clean":"2024-04-26T20:22:41.762202+0000","last_became_active":"2024-04-26T20:22:41.759034+0000","last_became_peered":"2024-04-26T20:22:41.759034+0000","last_unstale":"2024-04-26T20:22:41.762202+0000","last_undegraded":"2024-04-26T20:22:41.762202+0000","last_fullsized":"2024-04-26T20:22:41.762202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:19.607106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751655+0000","last_change":"2024-04-26T20:22:41.751655+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751655+0000","last_undegraded":"2024-04-26T20:22:41.751655+0000","last_fullsized":"2024-04-26T20:22:41.751655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761347+0000","last_change":"2024-04-26T20:22:41.760401+0000","last_active":"2024-04-26T20:22:41.761347+0000","last_peered":"2024-04-26T20:22:41.761347+0000","last_clean":"2024-04-26T20:22:41.761347+0000","last_became_active":"2024-04-26T20:22:41.760080+0000","last_became_peered":"2024-04-26T20:22:41.760080+0000","last_unstale":"2024-04-26T20:22:41.761347+0000","last_undegraded":"2024-04-26T20:22:41.761347+0000","last_fullsized":"2024-04-26T20:22:41.761347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:02.047370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744516+0000","last_change":"2024-04-26T20:22:41.744516+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744516+0000","last_undegraded":"2024-04-26T20:22:41.744516+0000","last_fullsized":"2024-04-26T20:22:41.744516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760372+0000","last_change":"2024-04-26T20:22:41.756976+0000","last_active":"2024-04-26T20:22:41.760372+0000","last_peered":"2024-04-26T20:22:41.760372+0000","last_clean":"2024-04-26T20:22:41.760372+0000","last_became_active":"2024-04-26T20:22:41.756639+0000","last_became_peered":"2024-04-26T20:22:41.756639+0000","last_unstale":"2024-04-26T20:22:41.760372+0000","last_undegraded":"2024-04-26T20:22:41.760372+0000","last_fullsized":"2024-04-26T20:22:41.760372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751992+0000","last_change":"2024-04-26T20:22:41.751992+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751992+0000","last_undegraded":"2024-04-26T20:22:41.751992+0000","last_fullsized":"2024-04-26T20:22:41.751992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.749878+0000","last_change":"2024-04-26T20:22:41.747756+0000","last_active":"2024-04-26T20:22:41.749878+0000","last_peered":"2024-04-26T20:22:41.749878+0000","last_clean":"2024-04-26T20:22:41.749878+0000","last_became_active":"2024-04-26T20:22:41.747460+0000","last_became_peered":"2024-04-26T20:22:41.747460+0000","last_unstale":"2024-04-26T20:22:41.749878+0000","last_undegraded":"2024-04-26T20:22:41.749878+0000","last_fullsized":"2024-04-26T20:22:41.749878+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.742836+0000","last_change":"2024-04-26T20:22:41.742836+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.742836+0000","last_undegraded":"2024-04-26T20:22:41.742836+0000","last_fullsized":"2024-04-26T20:22:41.742836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758269+0000","last_change":"2024-04-26T20:22:41.750670+0000","last_active":"2024-04-26T20:22:41.758269+0000","last_peered":"2024-04-26T20:22:41.758269+0000","last_clean":"2024-04-26T20:22:41.758269+0000","last_became_active":"2024-04-26T20:22:41.750317+0000","last_became_peered":"2024-04-26T20:22:41.750317+0000","last_unstale":"2024-04-26T20:22:41.758269+0000","last_undegraded":"2024-04-26T20:22:41.758269+0000","last_fullsized":"2024-04-26T20:22:41.758269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:55.932492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.747088+0000","last_change":"2024-04-26T20:22:41.747088+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.747088+0000","last_undegraded":"2024-04-26T20:22:41.747088+0000","last_fullsized":"2024-04-26T20:22:41.747088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.756385+0000","last_change":"2024-04-26T20:22:41.754948+0000","last_active":"2024-04-26T20:22:41.756385+0000","last_peered":"2024-04-26T20:22:41.756385+0000","last_clean":"2024-04-26T20:22:41.756385+0000","last_became_active":"2024-04-26T20:22:41.754664+0000","last_became_peered":"2024-04-26T20:22:41.754664+0000","last_unstale":"2024-04-26T20:22:41.756385+0000","last_undegraded":"2024-04-26T20:22:41.756385+0000","last_fullsized":"2024-04-26T20:22:41.756385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743378+0000","last_change":"2024-04-26T20:22:41.743378+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743378+0000","last_undegraded":"2024-04-26T20:22:41.743378+0000","last_fullsized":"2024-04-26T20:22:41.743378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760503+0000","last_change":"2024-04-26T20:22:41.756879+0000","last_active":"2024-04-26T20:22:41.760503+0000","last_peered":"2024-04-26T20:22:41.760503+0000","last_clean":"2024-04-26T20:22:41.760503+0000","last_became_active":"2024-04-26T20:22:41.756600+0000","last_became_peered":"2024-04-26T20:22:41.756600+0000","last_unstale":"2024-04-26T20:22:41.760503+0000","last_undegraded":"2024-04-26T20:22:41.760503+0000","last_fullsized":"2024-04-26T20:22:41.760503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750965+0000","last_change":"2024-04-26T20:22:41.750965+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750965+0000","last_undegraded":"2024-04-26T20:22:41.750965+0000","last_fullsized":"2024-04-26T20:22:41.750965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.759460+0000","last_change":"2024-04-26T20:22:41.756827+0000","last_active":"2024-04-26T20:22:41.759460+0000","last_peered":"2024-04-26T20:22:41.759460+0000","last_clean":"2024-04-26T20:22:41.759460+0000","last_became_active":"2024-04-26T20:22:41.756542+0000","last_became_peered":"2024-04-26T20:22:41.756542+0000","last_unstale":"2024-04-26T20:22:41.759460+0000","last_undegraded":"2024-04-26T20:22:41.759460+0000","last_fullsized":"2024-04-26T20:22:41.759460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:52.253630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748484+0000","last_change":"2024-04-26T20:22:41.748484+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748484+0000","last_undegraded":"2024-04-26T20:22:41.748484+0000","last_fullsized":"2024-04-26T20:22:41.748484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765234+0000","last_change":"2024-04-26T20:22:41.760981+0000","last_active":"2024-04-26T20:22:41.765234+0000","last_peered":"2024-04-26T20:22:41.765234+0000","last_clean":"2024-04-26T20:22:41.765234+0000","last_became_active":"2024-04-26T20:22:41.760700+0000","last_became_peered":"2024-04-26T20:22:41.760700+0000","last_unstale":"2024-04-26T20:22:41.765234+0000","last_undegraded":"2024-04-26T20:22:41.765234+0000","last_fullsized":"2024-04-26T20:22:41.765234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:20.612907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750327+0000","last_change":"2024-04-26T20:22:41.750327+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750327+0000","last_undegraded":"2024-04-26T20:22:41.750327+0000","last_fullsized":"2024-04-26T20:22:41.750327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760640+0000","last_change":"2024-04-26T20:22:41.755373+0000","last_active":"2024-04-26T20:22:41.760640+0000","last_peered":"2024-04-26T20:22:41.760640+0000","last_clean":"2024-04-26T20:22:41.760640+0000","last_became_active":"2024-04-26T20:22:41.755087+0000","last_became_peered":"2024-04-26T20:22:41.755087+0000","last_unstale":"2024-04-26T20:22:41.760640+0000","last_undegraded":"2024-04-26T20:22:41.760640+0000","last_fullsized":"2024-04-26T20:22:41.760640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.756741+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:22:41.756741+0000","last_peered":"2024-04-26T20:22:41.756741+0000","last_clean":"2024-04-26T20:22:41.756741+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:22:41.756741+0000","last_undegraded":"2024-04-26T20:22:41.756741+0000","last_fullsized":"2024-04-26T20:22:41.756741+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:22:47.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-26T20:22:47.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.747315+0000', 'last_change': '2024-04-26T20:22:41.747315+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.747315+0000', 'last_undegraded': '2024-04-26T20:22:41.747315+0000', 'last_fullsized': '2024-04-26T20:22:41.747315+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:22:47.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-26T20:22:47.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.751176+0000', 'last_change': '2024-04-26T20:22:41.751176+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.751176+0000', 'last_undegraded': '2024-04-26T20:22:41.751176+0000', 'last_fullsized': '2024-04-26T20:22:41.751176+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:22:47.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-26T20:22:47.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.749201+0000', 'last_change': '2024-04-26T20:22:41.749201+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.749201+0000', 'last_undegraded': '2024-04-26T20:22:41.749201+0000', 'last_fullsized': '2024-04-26T20:22:41.749201+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:22:47.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-26T20:22:47.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.745545+0000', 'last_change': '2024-04-26T20:22:41.745545+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.745545+0000', 'last_undegraded': '2024-04-26T20:22:41.745545+0000', 'last_fullsized': '2024-04-26T20:22:41.745545+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:22:47.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-26T20:22:47.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.745404+0000', 'last_change': '2024-04-26T20:22:41.745404+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.745404+0000', 'last_undegraded': '2024-04-26T20:22:41.745404+0000', 'last_fullsized': '2024-04-26T20:22:41.745404+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:22:47.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-26T20:22:47.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750104+0000', 'last_change': '2024-04-26T20:22:41.750104+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750104+0000', 'last_undegraded': '2024-04-26T20:22:41.750104+0000', 'last_fullsized': '2024-04-26T20:22:41.750104+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:22:47.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-26T20:22:47.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750969+0000', 'last_change': '2024-04-26T20:22:41.750969+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750969+0000', 'last_undegraded': '2024-04-26T20:22:41.750969+0000', 'last_fullsized': '2024-04-26T20:22:41.750969+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:22:47.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-26T20:22:47.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.753949+0000', 'last_change': '2024-04-26T20:22:41.753949+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.753949+0000', 'last_undegraded': '2024-04-26T20:22:41.753949+0000', 'last_fullsized': '2024-04-26T20:22:41.753949+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:22:47.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-26T20:22:47.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.749215+0000', 'last_change': '2024-04-26T20:22:41.749215+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.749215+0000', 'last_undegraded': '2024-04-26T20:22:41.749215+0000', 'last_fullsized': '2024-04-26T20:22:41.749215+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:22:47.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-26T20:22:47.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.744734+0000', 'last_change': '2024-04-26T20:22:41.744734+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.744734+0000', 'last_undegraded': '2024-04-26T20:22:41.744734+0000', 'last_fullsized': '2024-04-26T20:22:41.744734+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:22:47.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-26T20:22:47.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.751008+0000', 'last_change': '2024-04-26T20:22:41.751008+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.751008+0000', 'last_undegraded': '2024-04-26T20:22:41.751008+0000', 'last_fullsized': '2024-04-26T20:22:41.751008+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:22:47.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-26T20:22:47.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.751513+0000', 'last_change': '2024-04-26T20:22:41.751513+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.751513+0000', 'last_undegraded': '2024-04-26T20:22:41.751513+0000', 'last_fullsized': '2024-04-26T20:22:41.751513+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:22:47.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-26T20:22:47.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.744914+0000', 'last_change': '2024-04-26T20:22:41.744914+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.744914+0000', 'last_undegraded': '2024-04-26T20:22:41.744914+0000', 'last_fullsized': '2024-04-26T20:22:41.744914+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:22:47.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-26T20:22:47.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.749030+0000', 'last_change': '2024-04-26T20:22:41.749030+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.749030+0000', 'last_undegraded': '2024-04-26T20:22:41.749030+0000', 'last_fullsized': '2024-04-26T20:22:41.749030+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:22:47.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-26T20:22:47.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750437+0000', 'last_change': '2024-04-26T20:22:41.750437+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750437+0000', 'last_undegraded': '2024-04-26T20:22:41.750437+0000', 'last_fullsized': '2024-04-26T20:22:41.750437+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:22:47.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-26T20:22:47.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.751270+0000', 'last_change': '2024-04-26T20:22:41.751270+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.751270+0000', 'last_undegraded': '2024-04-26T20:22:41.751270+0000', 'last_fullsized': '2024-04-26T20:22:41.751270+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:22:47.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-26T20:22:47.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.751848+0000', 'last_change': '2024-04-26T20:22:41.751848+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.751848+0000', 'last_undegraded': '2024-04-26T20:22:41.751848+0000', 'last_fullsized': '2024-04-26T20:22:41.751848+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750736+0000', 'last_change': '2024-04-26T20:22:41.750736+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750736+0000', 'last_undegraded': '2024-04-26T20:22:41.750736+0000', 'last_fullsized': '2024-04-26T20:22:41.750736+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750525+0000', 'last_change': '2024-04-26T20:22:41.750525+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750525+0000', 'last_undegraded': '2024-04-26T20:22:41.750525+0000', 'last_fullsized': '2024-04-26T20:22:41.750525+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.744169+0000', 'last_change': '2024-04-26T20:22:41.744169+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.744169+0000', 'last_undegraded': '2024-04-26T20:22:41.744169+0000', 'last_fullsized': '2024-04-26T20:22:41.744169+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.748990+0000', 'last_change': '2024-04-26T20:22:41.748990+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.748990+0000', 'last_undegraded': '2024-04-26T20:22:41.748990+0000', 'last_fullsized': '2024-04-26T20:22:41.748990+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750071+0000', 'last_change': '2024-04-26T20:22:41.750071+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750071+0000', 'last_undegraded': '2024-04-26T20:22:41.750071+0000', 'last_fullsized': '2024-04-26T20:22:41.750071+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-26T20:22:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.744336+0000', 'last_change': '2024-04-26T20:22:41.744336+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.744336+0000', 'last_undegraded': '2024-04-26T20:22:41.744336+0000', 'last_fullsized': '2024-04-26T20:22:41.744336+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.749073+0000', 'last_change': '2024-04-26T20:22:41.749073+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.749073+0000', 'last_undegraded': '2024-04-26T20:22:41.749073+0000', 'last_fullsized': '2024-04-26T20:22:41.749073+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.745963+0000', 'last_change': '2024-04-26T20:22:41.745963+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.745963+0000', 'last_undegraded': '2024-04-26T20:22:41.745963+0000', 'last_fullsized': '2024-04-26T20:22:41.745963+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750267+0000', 'last_change': '2024-04-26T20:22:41.750267+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750267+0000', 'last_undegraded': '2024-04-26T20:22:41.750267+0000', 'last_fullsized': '2024-04-26T20:22:41.750267+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.744351+0000', 'last_change': '2024-04-26T20:22:41.744351+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.744351+0000', 'last_undegraded': '2024-04-26T20:22:41.744351+0000', 'last_fullsized': '2024-04-26T20:22:41.744351+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.748823+0000', 'last_change': '2024-04-26T20:22:41.748823+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.748823+0000', 'last_undegraded': '2024-04-26T20:22:41.748823+0000', 'last_fullsized': '2024-04-26T20:22:41.748823+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-26T20:22:47.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.748525+0000', 'last_change': '2024-04-26T20:22:41.748525+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.748525+0000', 'last_undegraded': '2024-04-26T20:22:41.748525+0000', 'last_fullsized': '2024-04-26T20:22:41.748525+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.752597+0000', 'last_change': '2024-04-26T20:22:41.752597+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.752597+0000', 'last_undegraded': '2024-04-26T20:22:41.752597+0000', 'last_fullsized': '2024-04-26T20:22:41.752597+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.746522+0000', 'last_change': '2024-04-26T20:22:41.746522+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.746522+0000', 'last_undegraded': '2024-04-26T20:22:41.746522+0000', 'last_fullsized': '2024-04-26T20:22:41.746522+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.753519+0000', 'last_change': '2024-04-26T20:22:41.753519+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.753519+0000', 'last_undegraded': '2024-04-26T20:22:41.753519+0000', 'last_fullsized': '2024-04-26T20:22:41.753519+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.744224+0000', 'last_change': '2024-04-26T20:22:41.744224+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.744224+0000', 'last_undegraded': '2024-04-26T20:22:41.744224+0000', 'last_fullsized': '2024-04-26T20:22:41.744224+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.746845+0000', 'last_change': '2024-04-26T20:22:41.746845+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.746845+0000', 'last_undegraded': '2024-04-26T20:22:41.746845+0000', 'last_fullsized': '2024-04-26T20:22:41.746845+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-26T20:22:47.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.749597+0000', 'last_change': '2024-04-26T20:22:41.749597+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.749597+0000', 'last_undegraded': '2024-04-26T20:22:41.749597+0000', 'last_fullsized': '2024-04-26T20:22:41.749597+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:22:47.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-26T20:22:47.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.743760+0000', 'last_change': '2024-04-26T20:22:41.743760+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.743760+0000', 'last_undegraded': '2024-04-26T20:22:41.743760+0000', 'last_fullsized': '2024-04-26T20:22:41.743760+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:22:47.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-26T20:22:47.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750686+0000', 'last_change': '2024-04-26T20:22:41.750686+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750686+0000', 'last_undegraded': '2024-04-26T20:22:41.750686+0000', 'last_fullsized': '2024-04-26T20:22:41.750686+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:22:47.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-26T20:22:47.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750653+0000', 'last_change': '2024-04-26T20:22:41.750653+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750653+0000', 'last_undegraded': '2024-04-26T20:22:41.750653+0000', 'last_fullsized': '2024-04-26T20:22:41.750653+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:22:47.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-26T20:22:47.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.749681+0000', 'last_change': '2024-04-26T20:22:41.749681+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.749681+0000', 'last_undegraded': '2024-04-26T20:22:41.749681+0000', 'last_fullsized': '2024-04-26T20:22:41.749681+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:22:47.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-26T20:22:47.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.743070+0000', 'last_change': '2024-04-26T20:22:41.743070+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.743070+0000', 'last_undegraded': '2024-04-26T20:22:41.743070+0000', 'last_fullsized': '2024-04-26T20:22:41.743070+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:22:47.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.748362+0000', 'last_change': '2024-04-26T20:22:41.748362+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.748362+0000', 'last_undegraded': '2024-04-26T20:22:41.748362+0000', 'last_fullsized': '2024-04-26T20:22:41.748362+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.749794+0000', 'last_change': '2024-04-26T20:22:41.749794+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.749794+0000', 'last_undegraded': '2024-04-26T20:22:41.749794+0000', 'last_fullsized': '2024-04-26T20:22:41.749794+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.749475+0000', 'last_change': '2024-04-26T20:22:41.749475+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.749475+0000', 'last_undegraded': '2024-04-26T20:22:41.749475+0000', 'last_fullsized': '2024-04-26T20:22:41.749475+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.749667+0000', 'last_change': '2024-04-26T20:22:41.749667+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.749667+0000', 'last_undegraded': '2024-04-26T20:22:41.749667+0000', 'last_fullsized': '2024-04-26T20:22:41.749667+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.743856+0000', 'last_change': '2024-04-26T20:22:41.743856+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.743856+0000', 'last_undegraded': '2024-04-26T20:22:41.743856+0000', 'last_fullsized': '2024-04-26T20:22:41.743856+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.748548+0000', 'last_change': '2024-04-26T20:22:41.748548+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.748548+0000', 'last_undegraded': '2024-04-26T20:22:41.748548+0000', 'last_fullsized': '2024-04-26T20:22:41.748548+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:22:47.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-26T20:22:47.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.753116+0000', 'last_change': '2024-04-26T20:22:41.753116+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.753116+0000', 'last_undegraded': '2024-04-26T20:22:41.753116+0000', 'last_fullsized': '2024-04-26T20:22:41.753116+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:22:47.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-26T20:22:47.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.745965+0000', 'last_change': '2024-04-26T20:22:41.745965+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.745965+0000', 'last_undegraded': '2024-04-26T20:22:41.745965+0000', 'last_fullsized': '2024-04-26T20:22:41.745965+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:22:47.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-26T20:22:47.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750914+0000', 'last_change': '2024-04-26T20:22:41.750914+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750914+0000', 'last_undegraded': '2024-04-26T20:22:41.750914+0000', 'last_fullsized': '2024-04-26T20:22:41.750914+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:22:47.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-26T20:22:47.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.752554+0000', 'last_change': '2024-04-26T20:22:41.752554+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.752554+0000', 'last_undegraded': '2024-04-26T20:22:41.752554+0000', 'last_fullsized': '2024-04-26T20:22:41.752554+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:22:47.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-26T20:22:47.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.745137+0000', 'last_change': '2024-04-26T20:22:41.745137+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.745137+0000', 'last_undegraded': '2024-04-26T20:22:41.745137+0000', 'last_fullsized': '2024-04-26T20:22:41.745137+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:22:47.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-26T20:22:47.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.744642+0000', 'last_change': '2024-04-26T20:22:41.744642+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.744642+0000', 'last_undegraded': '2024-04-26T20:22:41.744642+0000', 'last_fullsized': '2024-04-26T20:22:41.744642+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.743229+0000', 'last_change': '2024-04-26T20:22:41.743229+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.743229+0000', 'last_undegraded': '2024-04-26T20:22:41.743229+0000', 'last_fullsized': '2024-04-26T20:22:41.743229+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.743176+0000', 'last_change': '2024-04-26T20:22:41.743176+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.743176+0000', 'last_undegraded': '2024-04-26T20:22:41.743176+0000', 'last_fullsized': '2024-04-26T20:22:41.743176+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.745108+0000', 'last_change': '2024-04-26T20:22:41.745108+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.745108+0000', 'last_undegraded': '2024-04-26T20:22:41.745108+0000', 'last_fullsized': '2024-04-26T20:22:41.745108+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.751655+0000', 'last_change': '2024-04-26T20:22:41.751655+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.751655+0000', 'last_undegraded': '2024-04-26T20:22:41.751655+0000', 'last_fullsized': '2024-04-26T20:22:41.751655+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.744516+0000', 'last_change': '2024-04-26T20:22:41.744516+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.744516+0000', 'last_undegraded': '2024-04-26T20:22:41.744516+0000', 'last_fullsized': '2024-04-26T20:22:41.744516+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-26T20:22:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.751992+0000', 'last_change': '2024-04-26T20:22:41.751992+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.751992+0000', 'last_undegraded': '2024-04-26T20:22:41.751992+0000', 'last_fullsized': '2024-04-26T20:22:41.751992+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.742836+0000', 'last_change': '2024-04-26T20:22:41.742836+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.742836+0000', 'last_undegraded': '2024-04-26T20:22:41.742836+0000', 'last_fullsized': '2024-04-26T20:22:41.742836+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.747088+0000', 'last_change': '2024-04-26T20:22:41.747088+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.747088+0000', 'last_undegraded': '2024-04-26T20:22:41.747088+0000', 'last_fullsized': '2024-04-26T20:22:41.747088+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.743378+0000', 'last_change': '2024-04-26T20:22:41.743378+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.743378+0000', 'last_undegraded': '2024-04-26T20:22:41.743378+0000', 'last_fullsized': '2024-04-26T20:22:41.743378+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750965+0000', 'last_change': '2024-04-26T20:22:41.750965+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750965+0000', 'last_undegraded': '2024-04-26T20:22:41.750965+0000', 'last_fullsized': '2024-04-26T20:22:41.750965+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.748484+0000', 'last_change': '2024-04-26T20:22:41.748484+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.748484+0000', 'last_undegraded': '2024-04-26T20:22:41.748484+0000', 'last_fullsized': '2024-04-26T20:22:41.748484+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-26T20:22:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:22:41.750327+0000', 'last_change': '2024-04-26T20:22:41.750327+0000', 'last_active': '2024-04-26T20:22:41.730889+0000', 'last_peered': '2024-04-26T20:22:41.730889+0000', 'last_clean': '2024-04-26T20:22:41.730889+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:22:41.750327+0000', 'last_undegraded': '2024-04-26T20:22:41.750327+0000', 'last_fullsized': '2024-04-26T20:22:41.750327+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'last_clean_scrub_stamp': '2024-04-26T20:22:41.730889+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:22:47.495 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T20:22:47.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.618+0000 7fab74e86640 1 Processor -- start 2024-04-26T20:22:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.618+0000 7fab74e86640 1 -- start start 2024-04-26T20:22:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.619+0000 7fab74e86640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fab7015f560 0x7fab700d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.619+0000 7fab74e86640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab700d10a0 0x7fab700d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.619+0000 7fab74e86640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab7015f940 con 0x7fab700d21d0 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.619+0000 7fab74e86640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fab700d43a0 con 0x7fab7015f560 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.619+0000 7fab74e86640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fab700d3300 con 0x7fab700d10a0 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.619+0000 7fab6e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fab7015f560 0x7fab700d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.619+0000 7fab6ed9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fab700d21d0 0x7fab70163050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52592/0 (socket says 172.21.15.37:52592) 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.619+0000 7fab6ed9d640 1 -- 172.21.15.37:0/585029101 learned_addr learned my addr 172.21.15.37:0/585029101 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.620+0000 7fab6dd9b640 1 --2- 172.21.15.37:0/585029101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab700d10a0 0x7fab700d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.620+0000 7fab6e59c640 1 -- 172.21.15.37:0/585029101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab700d10a0 msgr2=0x7fab700d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.620+0000 7fab6e59c640 1 --2- 172.21.15.37:0/585029101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab700d10a0 0x7fab700d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.620+0000 7fab6e59c640 1 -- 172.21.15.37:0/585029101 >> v1:172.21.15.37:6789/0 conn(0x7fab700d21d0 legacy=0x7fab70163050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.620+0000 7fab6d59a640 1 -- 172.21.15.37:0/585029101 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 938371842 0 0) 0x7fab7015f940 con 0x7fab700d21d0 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.620+0000 7fab6e59c640 1 -- 172.21.15.37:0/585029101 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab700d35e0 con 0x7fab7015f560 2024-04-26T20:22:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.620+0000 7fab6dd9b640 1 --2- 172.21.15.37:0/585029101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab700d10a0 0x7fab700d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:47.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.620+0000 7fab6e59c640 1 --2- 172.21.15.37:0/585029101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7015f560 0x7fab700d0b60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fab680097a0 tx=0x7fab6802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e356ab05109eba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:47.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.621+0000 7fab6d59a640 1 -- 172.21.15.37:0/585029101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68037070 con 0x7fab7015f560 2024-04-26T20:22:47.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.621+0000 7fab6d59a640 1 -- 172.21.15.37:0/585029101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab68032440 con 0x7fab7015f560 2024-04-26T20:22:47.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.621+0000 7fab6d59a640 1 -- 172.21.15.37:0/585029101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68032810 con 0x7fab7015f560 2024-04-26T20:22:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.621+0000 7fab74e86640 1 -- 172.21.15.37:0/585029101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7015f560 msgr2=0x7fab700d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.621+0000 7fab74e86640 1 --2- 172.21.15.37:0/585029101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7015f560 0x7fab700d0b60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fab680097a0 tx=0x7fab6802a180 comp rx=0 tx=0).stop 2024-04-26T20:22:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.621+0000 7fab74e86640 1 -- 172.21.15.37:0/585029101 shutdown_connections 2024-04-26T20:22:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.621+0000 7fab74e86640 1 --2- 172.21.15.37:0/585029101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab700d10a0 0x7fab700d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.621+0000 7fab74e86640 1 --2- 172.21.15.37:0/585029101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7015f560 0x7fab700d0b60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.621+0000 7fab74e86640 1 -- 172.21.15.37:0/585029101 >> 172.21.15.37:0/585029101 conn(0x7fab7009e570 msgr2=0x7fab700d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.621+0000 7fab74e86640 1 -- 172.21.15.37:0/585029101 shutdown_connections 2024-04-26T20:22:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.621+0000 7fab74e86640 1 -- 172.21.15.37:0/585029101 wait complete. 2024-04-26T20:22:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.623+0000 7fab74e86640 1 Processor -- start 2024-04-26T20:22:47.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.623+0000 7fab74e86640 1 -- start start 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.623+0000 7fab74e86640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab700d10a0 0x7fab7015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.623+0000 7fab74e86640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab700d21d0 0x7fab700cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.623+0000 7fab74e86640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab7015f560 0x7fab70161c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.623+0000 7fab74e86640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab700d3e80 con 0x7fab700d10a0 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.623+0000 7fab74e86640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab700d3ba0 con 0x7fab7015f560 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.623+0000 7fab74e86640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fab700b6ac0 con 0x7fab700d21d0 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab700d10a0 0x7fab7015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab700d10a0 0x7fab7015a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35874/0 (socket says 172.21.15.37:35874) 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6e59c640 1 -- 172.21.15.37:0/1286278966 learned_addr learned my addr 172.21.15.37:0/1286278966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6dd9b640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab700d21d0 0x7fab700cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6ed9d640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab7015f560 0x7fab70161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6e59c640 1 -- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab700d21d0 msgr2=0x7fab700cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6e59c640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab700d21d0 0x7fab700cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6e59c640 1 -- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab7015f560 msgr2=0x7fab70161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6e59c640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab7015f560 0x7fab70161c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6e59c640 1 -- 172.21.15.37:0/1286278966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab700cbb80 con 0x7fab700d10a0 2024-04-26T20:22:47.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6ed9d640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab7015f560 0x7fab70161c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:47.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab6e59c640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab700d10a0 0x7fab7015a8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fab68035040 tx=0x7fab680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:47.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab66ffd640 1 -- 172.21.15.37:0/1286278966 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68037040 con 0x7fab700d10a0 2024-04-26T20:22:47.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab66ffd640 1 -- 172.21.15.37:0/1286278966 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab68030770 con 0x7fab700d10a0 2024-04-26T20:22:47.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab74e86640 1 -- 172.21.15.37:0/1286278966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7015aed0 con 0x7fab700d10a0 2024-04-26T20:22:47.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab66ffd640 1 -- 172.21.15.37:0/1286278966 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68030b80 con 0x7fab700d10a0 2024-04-26T20:22:47.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.624+0000 7fab74e86640 1 -- 172.21.15.37:0/1286278966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7015b490 con 0x7fab700d10a0 2024-04-26T20:22:47.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.625+0000 7fab74e86640 1 -- 172.21.15.37:0/1286278966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7006a360 con 0x7fab700d10a0 2024-04-26T20:22:47.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.626+0000 7fab66ffd640 1 -- 172.21.15.37:0/1286278966 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab68044020 con 0x7fab700d10a0 2024-04-26T20:22:47.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.626+0000 7fab66ffd640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab5006c8f0 0x7fab5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:47.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.627+0000 7fab66ffd640 1 -- 172.21.15.37:0/1286278966 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fab680abf70 con 0x7fab700d10a0 2024-04-26T20:22:47.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.627+0000 7fab6dd9b640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab5006c8f0 0x7fab5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:47.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.627+0000 7fab6dd9b640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab5006c8f0 0x7fab5006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab7015c4a0 tx=0x7fab58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:47.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.631+0000 7fab66ffd640 1 -- 172.21.15.37:0/1286278966 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab68032e20 con 0x7fab700d10a0 2024-04-26T20:22:47.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.886+0000 7fab74e86640 1 -- 172.21.15.37:0/1286278966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fab700baf80 con 0x7fab700d10a0 2024-04-26T20:22:47.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.887+0000 7fab66ffd640 1 -- 172.21.15.37:0/1286278966 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1581 (secure 0 0 0) 0x7fab6807a430 con 0x7fab700d10a0 2024-04-26T20:22:47.888 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:47.888 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":82,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1714162896,"num_in_osds":8,"osd_in_since":1714162887,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225288192,"bytes_avail":772868825088,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5097}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-26T20:22:39.345771+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T20:22:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 -- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab5006c8f0 msgr2=0x7fab5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab5006c8f0 0x7fab5006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab7015c4a0 tx=0x7fab58008040 comp rx=0 tx=0).stop 2024-04-26T20:22:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 -- 172.21.15.37:0/1286278966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab700d10a0 msgr2=0x7fab7015a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab700d10a0 0x7fab7015a8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fab68035040 tx=0x7fab680046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 -- 172.21.15.37:0/1286278966 shutdown_connections 2024-04-26T20:22:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab7015f560 0x7fab70161c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fab5006c8f0 0x7fab5006edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab700d21d0 0x7fab700cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 --2- 172.21.15.37:0/1286278966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab700d10a0 0x7fab7015a8f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 -- 172.21.15.37:0/1286278966 >> 172.21.15.37:0/1286278966 conn(0x7fab7009e570 msgr2=0x7fab70075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:47.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 -- 172.21.15.37:0/1286278966 shutdown_connections 2024-04-26T20:22:47.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:47.889+0000 7fab74e86640 1 -- 172.21.15.37:0/1286278966 wait complete. 2024-04-26T20:22:47.901 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T20:22:47.901 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:22:48.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.000+0000 7f104bc59640 1 Processor -- start 2024-04-26T20:22:48.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.000+0000 7f104bc59640 1 -- start start 2024-04-26T20:22:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.001+0000 7f104bc59640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f10440d2180 0x7f10440d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.001+0000 7f104bc59640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d3c80 0x7f10440b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.001+0000 7f104bc59640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10440d4090 con 0x7f104415f400 2024-04-26T20:22:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.001+0000 7f104bc59640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f10440d1290 con 0x7f10440d2180 2024-04-26T20:22:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.001+0000 7f104bc59640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f10440d3220 con 0x7f10440d3c80 2024-04-26T20:22:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.001+0000 7f10491f4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f10440d2180 0x7f10440d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.001+0000 7f10499f5640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f104415f400 0x7f10440d0b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52598/0 (socket says 172.21.15.37:52598) 2024-04-26T20:22:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.001+0000 7f10499f5640 1 -- 172.21.15.37:0/609730049 learned_addr learned my addr 172.21.15.37:0/609730049 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.002+0000 7f10491f4640 1 -- 172.21.15.37:0/609730049 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d3c80 msgr2=0x7f10440b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.002+0000 7f104a1f6640 1 --2- 172.21.15.37:0/609730049 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d3c80 0x7f10440b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:48.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.002+0000 7f10491f4640 1 --2- 172.21.15.37:0/609730049 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d3c80 0x7f10440b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:48.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.002+0000 7f10489f3640 1 -- 172.21.15.37:0/609730049 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2845038124 0 0) 0x7f1040001110 con 0x7f104415f400 2024-04-26T20:22:48.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.002+0000 7f10491f4640 1 -- 172.21.15.37:0/609730049 >> v1:172.21.15.37:6789/0 conn(0x7f104415f400 legacy=0x7f10440d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:48.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.002+0000 7f10491f4640 1 -- 172.21.15.37:0/609730049 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10440d5ef0 con 0x7f10440d2180 2024-04-26T20:22:48.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.002+0000 7f104a1f6640 1 --2- 172.21.15.37:0/609730049 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d3c80 0x7f10440b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:48.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.002+0000 7f10491f4640 1 --2- 172.21.15.37:0/609730049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10440d2180 0x7f10440d3740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f10340097a0 tx=0x7f103402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcb5d9cd9b26f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:48.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.003+0000 7f10489f3640 1 -- 172.21.15.37:0/609730049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034037070 con 0x7f10440d2180 2024-04-26T20:22:48.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.003+0000 7f10489f3640 1 -- 172.21.15.37:0/609730049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1034032440 con 0x7f10440d2180 2024-04-26T20:22:48.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.003+0000 7f10489f3640 1 -- 172.21.15.37:0/609730049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034032810 con 0x7f10440d2180 2024-04-26T20:22:48.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.003+0000 7f104bc59640 1 -- 172.21.15.37:0/609730049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10440d2180 msgr2=0x7f10440d3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:48.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.003+0000 7f104bc59640 1 --2- 172.21.15.37:0/609730049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10440d2180 0x7f10440d3740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f10340097a0 tx=0x7f103402a180 comp rx=0 tx=0).stop 2024-04-26T20:22:48.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.003+0000 7f104bc59640 1 -- 172.21.15.37:0/609730049 shutdown_connections 2024-04-26T20:22:48.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.003+0000 7f104bc59640 1 --2- 172.21.15.37:0/609730049 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d3c80 0x7f10440b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:48.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.003+0000 7f104bc59640 1 --2- 172.21.15.37:0/609730049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10440d2180 0x7f10440d3740 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:48.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.003+0000 7f104bc59640 1 -- 172.21.15.37:0/609730049 >> 172.21.15.37:0/609730049 conn(0x7f104409e510 msgr2=0x7f10440d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:48.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.003+0000 7f104bc59640 1 -- 172.21.15.37:0/609730049 shutdown_connections 2024-04-26T20:22:48.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.003+0000 7f104bc59640 1 -- 172.21.15.37:0/609730049 wait complete. 2024-04-26T20:22:48.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.004+0000 7f104bc59640 1 Processor -- start 2024-04-26T20:22:48.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.004+0000 7f104bc59640 1 -- start start 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.005+0000 7f104bc59640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d2180 0x7f1044151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.005+0000 7f104bc59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10440d3c80 0x7f104406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.005+0000 7f104bc59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f104415f400 0x7f104406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.005+0000 7f104bc59640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10440d0660 con 0x7f104415f400 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.005+0000 7f104bc59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10440d0380 con 0x7f10440d3c80 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.005+0000 7f104bc59640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f10440d25f0 con 0x7f10440d2180 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.005+0000 7f104a1f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f104415f400 0x7f104406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.005+0000 7f104a1f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f104415f400 0x7f104406d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35898/0 (socket says 172.21.15.37:35898) 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.005+0000 7f104a1f6640 1 -- 172.21.15.37:0/2799823063 learned_addr learned my addr 172.21.15.37:0/2799823063 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.005+0000 7f10499f5640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d2180 0x7f1044151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f104a1f6640 1 -- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d2180 msgr2=0x7f1044151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f10491f4640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10440d3c80 0x7f104406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:48.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f104a1f6640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d2180 0x7f1044151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f104a1f6640 1 -- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10440d3c80 msgr2=0x7f104406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f104a1f6640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10440d3c80 0x7f104406d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f104a1f6640 1 -- 172.21.15.37:0/2799823063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10440ce4c0 con 0x7f104415f400 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f10499f5640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d2180 0x7f1044151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f10491f4640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10440d3c80 0x7f104406d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f104a1f6640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f104415f400 0x7f104406d6e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f103c00cd80 tx=0x7f103c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f103a7fc640 1 -- 172.21.15.37:0/2799823063 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c00f680 con 0x7f104415f400 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f103a7fc640 1 -- 172.21.15.37:0/2799823063 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f103c00f840 con 0x7f104415f400 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f104bc59640 1 -- 172.21.15.37:0/2799823063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10440ce760 con 0x7f104415f400 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f103a7fc640 1 -- 172.21.15.37:0/2799823063 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c00fc30 con 0x7f104415f400 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.006+0000 7f104bc59640 1 -- 172.21.15.37:0/2799823063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10440ced20 con 0x7f104415f400 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.007+0000 7f104bc59640 1 -- 172.21.15.37:0/2799823063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10440d36c0 con 0x7f104415f400 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.007+0000 7f103a7fc640 1 -- 172.21.15.37:0/2799823063 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f103c0213f0 con 0x7f104415f400 2024-04-26T20:22:48.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.008+0000 7f103a7fc640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f102806c8f0 0x7f102806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.008+0000 7f103a7fc640 1 -- 172.21.15.37:0/2799823063 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f103c08c910 con 0x7f104415f400 2024-04-26T20:22:48.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.008+0000 7f10499f5640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f102806c8f0 0x7f102806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:48.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.009+0000 7f10499f5640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f102806c8f0 0x7f102806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f10400013b0 tx=0x7f1040009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:48.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.012+0000 7f103a7fc640 1 -- 172.21.15.37:0/2799823063 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f103c014030 con 0x7f104415f400 2024-04-26T20:22:48.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.192+0000 7f104bc59640 1 -- 172.21.15.37:0/2799823063 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10440693e0 con 0x7f102806c8f0 2024-04-26T20:22:48.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.199+0000 7f103a7fc640 1 -- 172.21.15.37:0/2799823063 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7f10440693e0 con 0x7f102806c8f0 2024-04-26T20:22:48.202 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:48.203 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 -- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f102806c8f0 msgr2=0x7f102806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f102806c8f0 0x7f102806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f10400013b0 tx=0x7f1040009040 comp rx=0 tx=0).stop 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 -- 172.21.15.37:0/2799823063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f104415f400 msgr2=0x7f104406d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f104415f400 0x7f104406d6e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f103c00cd80 tx=0x7f103c00a650 comp rx=0 tx=0).stop 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 -- 172.21.15.37:0/2799823063 shutdown_connections 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f104415f400 0x7f104406d6e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10440d3c80 0x7f104406d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f102806c8f0 0x7f102806edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 --2- 172.21.15.37:0/2799823063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10440d2180 0x7f1044151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 -- 172.21.15.37:0/2799823063 >> 172.21.15.37:0/2799823063 conn(0x7f104409e510 msgr2=0x7f104406e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 -- 172.21.15.37:0/2799823063 shutdown_connections 2024-04-26T20:22:48.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:48.201+0000 7f104bc59640 1 -- 172.21.15.37:0/2799823063 wait complete. 2024-04-26T20:22:48.213 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-04-26T20:22:47.347078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220008,"kb_used_data":3832,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754712,"statfs":{"total":773094113280,"available":772868825088,"internally_reserved":0,"allocated":3923968,"data_stored":2223996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001014"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.747315+0000","last_change":"2024-04-26T20:22:41.747315+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.747315+0000","last_undegraded":"2024-04-26T20:22:41.747315+0000","last_fullsized":"2024-04-26T20:22:41.747315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751176+0000","last_change":"2024-04-26T20:22:41.751176+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751176+0000","last_undegraded":"2024-04-26T20:22:41.751176+0000","last_fullsized":"2024-04-26T20:22:41.751176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749201+0000","last_change":"2024-04-26T20:22:41.749201+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749201+0000","last_undegraded":"2024-04-26T20:22:41.749201+0000","last_fullsized":"2024-04-26T20:22:41.749201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745545+0000","last_change":"2024-04-26T20:22:41.745545+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745545+0000","last_undegraded":"2024-04-26T20:22:41.745545+0000","last_fullsized":"2024-04-26T20:22:41.745545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745404+0000","last_change":"2024-04-26T20:22:41.745404+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745404+0000","last_undegraded":"2024-04-26T20:22:41.745404+0000","last_fullsized":"2024-04-26T20:22:41.745404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750104+0000","last_change":"2024-04-26T20:22:41.750104+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750104+0000","last_undegraded":"2024-04-26T20:22:41.750104+0000","last_fullsized":"2024-04-26T20:22:41.750104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750969+0000","last_change":"2024-04-26T20:22:41.750969+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750969+0000","last_undegraded":"2024-04-26T20:22:41.750969+0000","last_fullsized":"2024-04-26T20:22:41.750969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753949+0000","last_change":"2024-04-26T20:22:41.753949+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753949+0000","last_undegraded":"2024-04-26T20:22:41.753949+0000","last_fullsized":"2024-04-26T20:22:41.753949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749215+0000","last_change":"2024-04-26T20:22:41.749215+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749215+0000","last_undegraded":"2024-04-26T20:22:41.749215+0000","last_fullsized":"2024-04-26T20:22:41.749215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744734+0000","last_change":"2024-04-26T20:22:41.744734+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744734+0000","last_undegraded":"2024-04-26T20:22:41.744734+0000","last_fullsized":"2024-04-26T20:22:41.744734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751008+0000","last_change":"2024-04-26T20:22:41.751008+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751008+0000","last_undegraded":"2024-04-26T20:22:41.751008+0000","last_fullsized":"2024-04-26T20:22:41.751008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751513+0000","last_change":"2024-04-26T20:22:41.751513+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751513+0000","last_undegraded":"2024-04-26T20:22:41.751513+0000","last_fullsized":"2024-04-26T20:22:41.751513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744914+0000","last_change":"2024-04-26T20:22:41.744914+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744914+0000","last_undegraded":"2024-04-26T20:22:41.744914+0000","last_fullsized":"2024-04-26T20:22:41.744914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749030+0000","last_change":"2024-04-26T20:22:41.749030+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749030+0000","last_undegraded":"2024-04-26T20:22:41.749030+0000","last_fullsized":"2024-04-26T20:22:41.749030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750437+0000","last_change":"2024-04-26T20:22:41.750437+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750437+0000","last_undegraded":"2024-04-26T20:22:41.750437+0000","last_fullsized":"2024-04-26T20:22:41.750437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751270+0000","last_change":"2024-04-26T20:22:41.751270+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751270+0000","last_undegraded":"2024-04-26T20:22:41.751270+0000","last_fullsized":"2024-04-26T20:22:41.751270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751848+0000","last_change":"2024-04-26T20:22:41.751848+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751848+0000","last_undegraded":"2024-04-26T20:22:41.751848+0000","last_fullsized":"2024-04-26T20:22:41.751848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750736+0000","last_change":"2024-04-26T20:22:41.750736+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750736+0000","last_undegraded":"2024-04-26T20:22:41.750736+0000","last_fullsized":"2024-04-26T20:22:41.750736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750525+0000","last_change":"2024-04-26T20:22:41.750525+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750525+0000","last_undegraded":"2024-04-26T20:22:41.750525+0000","last_fullsized":"2024-04-26T20:22:41.750525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744169+0000","last_change":"2024-04-26T20:22:41.744169+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744169+0000","last_undegraded":"2024-04-26T20:22:41.744169+0000","last_fullsized":"2024-04-26T20:22:41.744169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748990+0000","last_change":"2024-04-26T20:22:41.748990+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748990+0000","last_undegraded":"2024-04-26T20:22:41.748990+0000","last_fullsized":"2024-04-26T20:22:41.748990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750071+0000","last_change":"2024-04-26T20:22:41.750071+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750071+0000","last_undegraded":"2024-04-26T20:22:41.750071+0000","last_fullsized":"2024-04-26T20:22:41.750071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744336+0000","last_change":"2024-04-26T20:22:41.744336+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744336+0000","last_undegraded":"2024-04-26T20:22:41.744336+0000","last_fullsized":"2024-04-26T20:22:41.744336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749073+0000","last_change":"2024-04-26T20:22:41.749073+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749073+0000","last_undegraded":"2024-04-26T20:22:41.749073+0000","last_fullsized":"2024-04-26T20:22:41.749073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745963+0000","last_change":"2024-04-26T20:22:41.745963+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745963+0000","last_undegraded":"2024-04-26T20:22:41.745963+0000","last_fullsized":"2024-04-26T20:22:41.745963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750267+0000","last_change":"2024-04-26T20:22:41.750267+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750267+0000","last_undegraded":"2024-04-26T20:22:41.750267+0000","last_fullsized":"2024-04-26T20:22:41.750267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744351+0000","last_change":"2024-04-26T20:22:41.744351+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744351+0000","last_undegraded":"2024-04-26T20:22:41.744351+0000","last_fullsized":"2024-04-26T20:22:41.744351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748823+0000","last_change":"2024-04-26T20:22:41.748823+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748823+0000","last_undegraded":"2024-04-26T20:22:41.748823+0000","last_fullsized":"2024-04-26T20:22:41.748823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748525+0000","last_change":"2024-04-26T20:22:41.748525+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748525+0000","last_undegraded":"2024-04-26T20:22:41.748525+0000","last_fullsized":"2024-04-26T20:22:41.748525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.752597+0000","last_change":"2024-04-26T20:22:41.752597+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.752597+0000","last_undegraded":"2024-04-26T20:22:41.752597+0000","last_fullsized":"2024-04-26T20:22:41.752597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.746522+0000","last_change":"2024-04-26T20:22:41.746522+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.746522+0000","last_undegraded":"2024-04-26T20:22:41.746522+0000","last_fullsized":"2024-04-26T20:22:41.746522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753519+0000","last_change":"2024-04-26T20:22:41.753519+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753519+0000","last_undegraded":"2024-04-26T20:22:41.753519+0000","last_fullsized":"2024-04-26T20:22:41.753519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744224+0000","last_change":"2024-04-26T20:22:41.744224+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744224+0000","last_undegraded":"2024-04-26T20:22:41.744224+0000","last_fullsized":"2024-04-26T20:22:41.744224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760557+0000","last_change":"2024-04-26T20:22:41.756667+0000","last_active":"2024-04-26T20:22:41.760557+0000","last_peered":"2024-04-26T20:22:41.760557+0000","last_clean":"2024-04-26T20:22:41.760557+0000","last_became_active":"2024-04-26T20:22:41.756385+0000","last_became_peered":"2024-04-26T20:22:41.756385+0000","last_unstale":"2024-04-26T20:22:41.760557+0000","last_undegraded":"2024-04-26T20:22:41.760557+0000","last_fullsized":"2024-04-26T20:22:41.760557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.746845+0000","last_change":"2024-04-26T20:22:41.746845+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.746845+0000","last_undegraded":"2024-04-26T20:22:41.746845+0000","last_fullsized":"2024-04-26T20:22:41.746845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764642+0000","last_change":"2024-04-26T20:22:41.758415+0000","last_active":"2024-04-26T20:22:41.764642+0000","last_peered":"2024-04-26T20:22:41.764642+0000","last_clean":"2024-04-26T20:22:41.764642+0000","last_became_active":"2024-04-26T20:22:41.757712+0000","last_became_peered":"2024-04-26T20:22:41.757712+0000","last_unstale":"2024-04-26T20:22:41.764642+0000","last_undegraded":"2024-04-26T20:22:41.764642+0000","last_fullsized":"2024-04-26T20:22:41.764642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:38.972810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749597+0000","last_change":"2024-04-26T20:22:41.749597+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749597+0000","last_undegraded":"2024-04-26T20:22:41.749597+0000","last_fullsized":"2024-04-26T20:22:41.749597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764752+0000","last_change":"2024-04-26T20:22:41.759486+0000","last_active":"2024-04-26T20:22:41.764752+0000","last_peered":"2024-04-26T20:22:41.764752+0000","last_clean":"2024-04-26T20:22:41.764752+0000","last_became_active":"2024-04-26T20:22:41.759076+0000","last_became_peered":"2024-04-26T20:22:41.759076+0000","last_unstale":"2024-04-26T20:22:41.764752+0000","last_undegraded":"2024-04-26T20:22:41.764752+0000","last_fullsized":"2024-04-26T20:22:41.764752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:27.337131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743760+0000","last_change":"2024-04-26T20:22:41.743760+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743760+0000","last_undegraded":"2024-04-26T20:22:41.743760+0000","last_fullsized":"2024-04-26T20:22:41.743760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758241+0000","last_change":"2024-04-26T20:22:41.754766+0000","last_active":"2024-04-26T20:22:41.758241+0000","last_peered":"2024-04-26T20:22:41.758241+0000","last_clean":"2024-04-26T20:22:41.758241+0000","last_became_active":"2024-04-26T20:22:41.754427+0000","last_became_peered":"2024-04-26T20:22:41.754427+0000","last_unstale":"2024-04-26T20:22:41.758241+0000","last_undegraded":"2024-04-26T20:22:41.758241+0000","last_fullsized":"2024-04-26T20:22:41.758241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:52.253630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750686+0000","last_change":"2024-04-26T20:22:41.750686+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750686+0000","last_undegraded":"2024-04-26T20:22:41.750686+0000","last_fullsized":"2024-04-26T20:22:41.750686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762211+0000","last_change":"2024-04-26T20:22:41.759951+0000","last_active":"2024-04-26T20:22:41.762211+0000","last_peered":"2024-04-26T20:22:41.762211+0000","last_clean":"2024-04-26T20:22:41.762211+0000","last_became_active":"2024-04-26T20:22:41.759681+0000","last_became_peered":"2024-04-26T20:22:41.759681+0000","last_unstale":"2024-04-26T20:22:41.762211+0000","last_undegraded":"2024-04-26T20:22:41.762211+0000","last_fullsized":"2024-04-26T20:22:41.762211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:49.082266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750653+0000","last_change":"2024-04-26T20:22:41.750653+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750653+0000","last_undegraded":"2024-04-26T20:22:41.750653+0000","last_fullsized":"2024-04-26T20:22:41.750653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761855+0000","last_change":"2024-04-26T20:22:41.758461+0000","last_active":"2024-04-26T20:22:41.761855+0000","last_peered":"2024-04-26T20:22:41.761855+0000","last_clean":"2024-04-26T20:22:41.761855+0000","last_became_active":"2024-04-26T20:22:41.758166+0000","last_became_peered":"2024-04-26T20:22:41.758166+0000","last_unstale":"2024-04-26T20:22:41.761855+0000","last_undegraded":"2024-04-26T20:22:41.761855+0000","last_fullsized":"2024-04-26T20:22:41.761855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:24.346362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749681+0000","last_change":"2024-04-26T20:22:41.749681+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749681+0000","last_undegraded":"2024-04-26T20:22:41.749681+0000","last_fullsized":"2024-04-26T20:22:41.749681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760841+0000","last_change":"2024-04-26T20:22:41.754578+0000","last_active":"2024-04-26T20:22:41.760841+0000","last_peered":"2024-04-26T20:22:41.760841+0000","last_clean":"2024-04-26T20:22:41.760841+0000","last_became_active":"2024-04-26T20:22:41.754286+0000","last_became_peered":"2024-04-26T20:22:41.754286+0000","last_unstale":"2024-04-26T20:22:41.760841+0000","last_undegraded":"2024-04-26T20:22:41.760841+0000","last_fullsized":"2024-04-26T20:22:41.760841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743070+0000","last_change":"2024-04-26T20:22:41.743070+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743070+0000","last_undegraded":"2024-04-26T20:22:41.743070+0000","last_fullsized":"2024-04-26T20:22:41.743070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765354+0000","last_change":"2024-04-26T20:22:41.762053+0000","last_active":"2024-04-26T20:22:41.765354+0000","last_peered":"2024-04-26T20:22:41.765354+0000","last_clean":"2024-04-26T20:22:41.765354+0000","last_became_active":"2024-04-26T20:22:41.761753+0000","last_became_peered":"2024-04-26T20:22:41.761753+0000","last_unstale":"2024-04-26T20:22:41.765354+0000","last_undegraded":"2024-04-26T20:22:41.765354+0000","last_fullsized":"2024-04-26T20:22:41.765354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:36.559062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748362+0000","last_change":"2024-04-26T20:22:41.748362+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748362+0000","last_undegraded":"2024-04-26T20:22:41.748362+0000","last_fullsized":"2024-04-26T20:22:41.748362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765262+0000","last_change":"2024-04-26T20:22:41.761862+0000","last_active":"2024-04-26T20:22:41.765262+0000","last_peered":"2024-04-26T20:22:41.765262+0000","last_clean":"2024-04-26T20:22:41.765262+0000","last_became_active":"2024-04-26T20:22:41.761611+0000","last_became_peered":"2024-04-26T20:22:41.761611+0000","last_unstale":"2024-04-26T20:22:41.765262+0000","last_undegraded":"2024-04-26T20:22:41.765262+0000","last_fullsized":"2024-04-26T20:22:41.765262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:02.047370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749794+0000","last_change":"2024-04-26T20:22:41.749794+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749794+0000","last_undegraded":"2024-04-26T20:22:41.749794+0000","last_fullsized":"2024-04-26T20:22:41.749794+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764613+0000","last_change":"2024-04-26T20:22:41.757671+0000","last_active":"2024-04-26T20:22:41.764613+0000","last_peered":"2024-04-26T20:22:41.764613+0000","last_clean":"2024-04-26T20:22:41.764613+0000","last_became_active":"2024-04-26T20:22:41.757277+0000","last_became_peered":"2024-04-26T20:22:41.757277+0000","last_unstale":"2024-04-26T20:22:41.764613+0000","last_undegraded":"2024-04-26T20:22:41.764613+0000","last_fullsized":"2024-04-26T20:22:41.764613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749475+0000","last_change":"2024-04-26T20:22:41.749475+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749475+0000","last_undegraded":"2024-04-26T20:22:41.749475+0000","last_fullsized":"2024-04-26T20:22:41.749475+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762294+0000","last_change":"2024-04-26T20:22:41.760284+0000","last_active":"2024-04-26T20:22:41.762294+0000","last_peered":"2024-04-26T20:22:41.762294+0000","last_clean":"2024-04-26T20:22:41.762294+0000","last_became_active":"2024-04-26T20:22:41.759981+0000","last_became_peered":"2024-04-26T20:22:41.759981+0000","last_unstale":"2024-04-26T20:22:41.762294+0000","last_undegraded":"2024-04-26T20:22:41.762294+0000","last_fullsized":"2024-04-26T20:22:41.762294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:20.612907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.749667+0000","last_change":"2024-04-26T20:22:41.749667+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.749667+0000","last_undegraded":"2024-04-26T20:22:41.749667+0000","last_fullsized":"2024-04-26T20:22:41.749667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764722+0000","last_change":"2024-04-26T20:22:41.759409+0000","last_active":"2024-04-26T20:22:41.764722+0000","last_peered":"2024-04-26T20:22:41.764722+0000","last_clean":"2024-04-26T20:22:41.764722+0000","last_became_active":"2024-04-26T20:22:41.758974+0000","last_became_peered":"2024-04-26T20:22:41.758974+0000","last_unstale":"2024-04-26T20:22:41.764722+0000","last_undegraded":"2024-04-26T20:22:41.764722+0000","last_fullsized":"2024-04-26T20:22:41.764722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:38.972810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743856+0000","last_change":"2024-04-26T20:22:41.743856+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743856+0000","last_undegraded":"2024-04-26T20:22:41.743856+0000","last_fullsized":"2024-04-26T20:22:41.743856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762234+0000","last_change":"2024-04-26T20:22:41.760955+0000","last_active":"2024-04-26T20:22:41.762234+0000","last_peered":"2024-04-26T20:22:41.762234+0000","last_clean":"2024-04-26T20:22:41.762234+0000","last_became_active":"2024-04-26T20:22:41.760658+0000","last_became_peered":"2024-04-26T20:22:41.760658+0000","last_unstale":"2024-04-26T20:22:41.762234+0000","last_undegraded":"2024-04-26T20:22:41.762234+0000","last_fullsized":"2024-04-26T20:22:41.762234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:42.223536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748548+0000","last_change":"2024-04-26T20:22:41.748548+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748548+0000","last_undegraded":"2024-04-26T20:22:41.748548+0000","last_fullsized":"2024-04-26T20:22:41.748548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758419+0000","last_change":"2024-04-26T20:22:41.751412+0000","last_active":"2024-04-26T20:22:41.758419+0000","last_peered":"2024-04-26T20:22:41.758419+0000","last_clean":"2024-04-26T20:22:41.758419+0000","last_became_active":"2024-04-26T20:22:41.751102+0000","last_became_peered":"2024-04-26T20:22:41.751102+0000","last_unstale":"2024-04-26T20:22:41.758419+0000","last_undegraded":"2024-04-26T20:22:41.758419+0000","last_fullsized":"2024-04-26T20:22:41.758419+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.753116+0000","last_change":"2024-04-26T20:22:41.753116+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.753116+0000","last_undegraded":"2024-04-26T20:22:41.753116+0000","last_fullsized":"2024-04-26T20:22:41.753116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765217+0000","last_change":"2024-04-26T20:22:41.760709+0000","last_active":"2024-04-26T20:22:41.765217+0000","last_peered":"2024-04-26T20:22:41.765217+0000","last_clean":"2024-04-26T20:22:41.765217+0000","last_became_active":"2024-04-26T20:22:41.760281+0000","last_became_peered":"2024-04-26T20:22:41.760281+0000","last_unstale":"2024-04-26T20:22:41.765217+0000","last_undegraded":"2024-04-26T20:22:41.765217+0000","last_fullsized":"2024-04-26T20:22:41.765217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:20.522370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745965+0000","last_change":"2024-04-26T20:22:41.745965+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745965+0000","last_undegraded":"2024-04-26T20:22:41.745965+0000","last_fullsized":"2024-04-26T20:22:41.745965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.764753+0000","last_change":"2024-04-26T20:22:41.759620+0000","last_active":"2024-04-26T20:22:41.764753+0000","last_peered":"2024-04-26T20:22:41.764753+0000","last_clean":"2024-04-26T20:22:41.764753+0000","last_became_active":"2024-04-26T20:22:41.759154+0000","last_became_peered":"2024-04-26T20:22:41.759154+0000","last_unstale":"2024-04-26T20:22:41.764753+0000","last_undegraded":"2024-04-26T20:22:41.764753+0000","last_fullsized":"2024-04-26T20:22:41.764753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:42.413094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750914+0000","last_change":"2024-04-26T20:22:41.750914+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750914+0000","last_undegraded":"2024-04-26T20:22:41.750914+0000","last_fullsized":"2024-04-26T20:22:41.750914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761309+0000","last_change":"2024-04-26T20:22:41.758134+0000","last_active":"2024-04-26T20:22:41.761309+0000","last_peered":"2024-04-26T20:22:41.761309+0000","last_clean":"2024-04-26T20:22:41.761309+0000","last_became_active":"2024-04-26T20:22:41.757853+0000","last_became_peered":"2024-04-26T20:22:41.757853+0000","last_unstale":"2024-04-26T20:22:41.761309+0000","last_undegraded":"2024-04-26T20:22:41.761309+0000","last_fullsized":"2024-04-26T20:22:41.761309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.752554+0000","last_change":"2024-04-26T20:22:41.752554+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.752554+0000","last_undegraded":"2024-04-26T20:22:41.752554+0000","last_fullsized":"2024-04-26T20:22:41.752554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758900+0000","last_change":"2024-04-26T20:22:41.756567+0000","last_active":"2024-04-26T20:22:41.758900+0000","last_peered":"2024-04-26T20:22:41.758900+0000","last_clean":"2024-04-26T20:22:41.758900+0000","last_became_active":"2024-04-26T20:22:41.756259+0000","last_became_peered":"2024-04-26T20:22:41.756259+0000","last_unstale":"2024-04-26T20:22:41.758900+0000","last_undegraded":"2024-04-26T20:22:41.758900+0000","last_fullsized":"2024-04-26T20:22:41.758900+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:19.518020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745137+0000","last_change":"2024-04-26T20:22:41.745137+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745137+0000","last_undegraded":"2024-04-26T20:22:41.745137+0000","last_fullsized":"2024-04-26T20:22:41.745137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761886+0000","last_change":"2024-04-26T20:22:41.757636+0000","last_active":"2024-04-26T20:22:41.761886+0000","last_peered":"2024-04-26T20:22:41.761886+0000","last_clean":"2024-04-26T20:22:41.761886+0000","last_became_active":"2024-04-26T20:22:41.757329+0000","last_became_peered":"2024-04-26T20:22:41.757329+0000","last_unstale":"2024-04-26T20:22:41.761886+0000","last_undegraded":"2024-04-26T20:22:41.761886+0000","last_fullsized":"2024-04-26T20:22:41.761886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:38.757083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744642+0000","last_change":"2024-04-26T20:22:41.744642+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744642+0000","last_undegraded":"2024-04-26T20:22:41.744642+0000","last_fullsized":"2024-04-26T20:22:41.744642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762077+0000","last_change":"2024-04-26T20:22:41.759343+0000","last_active":"2024-04-26T20:22:41.762077+0000","last_peered":"2024-04-26T20:22:41.762077+0000","last_clean":"2024-04-26T20:22:41.762077+0000","last_became_active":"2024-04-26T20:22:41.758924+0000","last_became_peered":"2024-04-26T20:22:41.758924+0000","last_unstale":"2024-04-26T20:22:41.762077+0000","last_undegraded":"2024-04-26T20:22:41.762077+0000","last_fullsized":"2024-04-26T20:22:41.762077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.731025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743229+0000","last_change":"2024-04-26T20:22:41.743229+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743229+0000","last_undegraded":"2024-04-26T20:22:41.743229+0000","last_fullsized":"2024-04-26T20:22:41.743229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.749913+0000","last_change":"2024-04-26T20:22:41.747174+0000","last_active":"2024-04-26T20:22:41.749913+0000","last_peered":"2024-04-26T20:22:41.749913+0000","last_clean":"2024-04-26T20:22:41.749913+0000","last_became_active":"2024-04-26T20:22:41.746153+0000","last_became_peered":"2024-04-26T20:22:41.746153+0000","last_unstale":"2024-04-26T20:22:41.749913+0000","last_undegraded":"2024-04-26T20:22:41.749913+0000","last_fullsized":"2024-04-26T20:22:41.749913+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743176+0000","last_change":"2024-04-26T20:22:41.743176+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743176+0000","last_undegraded":"2024-04-26T20:22:41.743176+0000","last_fullsized":"2024-04-26T20:22:41.743176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760527+0000","last_change":"2024-04-26T20:22:41.755144+0000","last_active":"2024-04-26T20:22:41.760527+0000","last_peered":"2024-04-26T20:22:41.760527+0000","last_clean":"2024-04-26T20:22:41.760527+0000","last_became_active":"2024-04-26T20:22:41.754847+0000","last_became_peered":"2024-04-26T20:22:41.754847+0000","last_unstale":"2024-04-26T20:22:41.760527+0000","last_undegraded":"2024-04-26T20:22:41.760527+0000","last_fullsized":"2024-04-26T20:22:41.760527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.745108+0000","last_change":"2024-04-26T20:22:41.745108+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.745108+0000","last_undegraded":"2024-04-26T20:22:41.745108+0000","last_fullsized":"2024-04-26T20:22:41.745108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.762202+0000","last_change":"2024-04-26T20:22:41.759321+0000","last_active":"2024-04-26T20:22:41.762202+0000","last_peered":"2024-04-26T20:22:41.762202+0000","last_clean":"2024-04-26T20:22:41.762202+0000","last_became_active":"2024-04-26T20:22:41.759034+0000","last_became_peered":"2024-04-26T20:22:41.759034+0000","last_unstale":"2024-04-26T20:22:41.762202+0000","last_undegraded":"2024-04-26T20:22:41.762202+0000","last_fullsized":"2024-04-26T20:22:41.762202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:19.607106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751655+0000","last_change":"2024-04-26T20:22:41.751655+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751655+0000","last_undegraded":"2024-04-26T20:22:41.751655+0000","last_fullsized":"2024-04-26T20:22:41.751655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.761347+0000","last_change":"2024-04-26T20:22:41.760401+0000","last_active":"2024-04-26T20:22:41.761347+0000","last_peered":"2024-04-26T20:22:41.761347+0000","last_clean":"2024-04-26T20:22:41.761347+0000","last_became_active":"2024-04-26T20:22:41.760080+0000","last_became_peered":"2024-04-26T20:22:41.760080+0000","last_unstale":"2024-04-26T20:22:41.761347+0000","last_undegraded":"2024-04-26T20:22:41.761347+0000","last_fullsized":"2024-04-26T20:22:41.761347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:02.047370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.744516+0000","last_change":"2024-04-26T20:22:41.744516+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.744516+0000","last_undegraded":"2024-04-26T20:22:41.744516+0000","last_fullsized":"2024-04-26T20:22:41.744516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760372+0000","last_change":"2024-04-26T20:22:41.756976+0000","last_active":"2024-04-26T20:22:41.760372+0000","last_peered":"2024-04-26T20:22:41.760372+0000","last_clean":"2024-04-26T20:22:41.760372+0000","last_became_active":"2024-04-26T20:22:41.756639+0000","last_became_peered":"2024-04-26T20:22:41.756639+0000","last_unstale":"2024-04-26T20:22:41.760372+0000","last_undegraded":"2024-04-26T20:22:41.760372+0000","last_fullsized":"2024-04-26T20:22:41.760372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.751992+0000","last_change":"2024-04-26T20:22:41.751992+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.751992+0000","last_undegraded":"2024-04-26T20:22:41.751992+0000","last_fullsized":"2024-04-26T20:22:41.751992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.749878+0000","last_change":"2024-04-26T20:22:41.747756+0000","last_active":"2024-04-26T20:22:41.749878+0000","last_peered":"2024-04-26T20:22:41.749878+0000","last_clean":"2024-04-26T20:22:41.749878+0000","last_became_active":"2024-04-26T20:22:41.747460+0000","last_became_peered":"2024-04-26T20:22:41.747460+0000","last_unstale":"2024-04-26T20:22:41.749878+0000","last_undegraded":"2024-04-26T20:22:41.749878+0000","last_fullsized":"2024-04-26T20:22:41.749878+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.742836+0000","last_change":"2024-04-26T20:22:41.742836+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.742836+0000","last_undegraded":"2024-04-26T20:22:41.742836+0000","last_fullsized":"2024-04-26T20:22:41.742836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.758269+0000","last_change":"2024-04-26T20:22:41.750670+0000","last_active":"2024-04-26T20:22:41.758269+0000","last_peered":"2024-04-26T20:22:41.758269+0000","last_clean":"2024-04-26T20:22:41.758269+0000","last_became_active":"2024-04-26T20:22:41.750317+0000","last_became_peered":"2024-04-26T20:22:41.750317+0000","last_unstale":"2024-04-26T20:22:41.758269+0000","last_undegraded":"2024-04-26T20:22:41.758269+0000","last_fullsized":"2024-04-26T20:22:41.758269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:55.932492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.747088+0000","last_change":"2024-04-26T20:22:41.747088+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.747088+0000","last_undegraded":"2024-04-26T20:22:41.747088+0000","last_fullsized":"2024-04-26T20:22:41.747088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.756385+0000","last_change":"2024-04-26T20:22:41.754948+0000","last_active":"2024-04-26T20:22:41.756385+0000","last_peered":"2024-04-26T20:22:41.756385+0000","last_clean":"2024-04-26T20:22:41.756385+0000","last_became_active":"2024-04-26T20:22:41.754664+0000","last_became_peered":"2024-04-26T20:22:41.754664+0000","last_unstale":"2024-04-26T20:22:41.756385+0000","last_undegraded":"2024-04-26T20:22:41.756385+0000","last_fullsized":"2024-04-26T20:22:41.756385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.743378+0000","last_change":"2024-04-26T20:22:41.743378+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.743378+0000","last_undegraded":"2024-04-26T20:22:41.743378+0000","last_fullsized":"2024-04-26T20:22:41.743378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760503+0000","last_change":"2024-04-26T20:22:41.756879+0000","last_active":"2024-04-26T20:22:41.760503+0000","last_peered":"2024-04-26T20:22:41.760503+0000","last_clean":"2024-04-26T20:22:41.760503+0000","last_became_active":"2024-04-26T20:22:41.756600+0000","last_became_peered":"2024-04-26T20:22:41.756600+0000","last_unstale":"2024-04-26T20:22:41.760503+0000","last_undegraded":"2024-04-26T20:22:41.760503+0000","last_fullsized":"2024-04-26T20:22:41.760503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750965+0000","last_change":"2024-04-26T20:22:41.750965+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750965+0000","last_undegraded":"2024-04-26T20:22:41.750965+0000","last_fullsized":"2024-04-26T20:22:41.750965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.759460+0000","last_change":"2024-04-26T20:22:41.756827+0000","last_active":"2024-04-26T20:22:41.759460+0000","last_peered":"2024-04-26T20:22:41.759460+0000","last_clean":"2024-04-26T20:22:41.759460+0000","last_became_active":"2024-04-26T20:22:41.756542+0000","last_became_peered":"2024-04-26T20:22:41.756542+0000","last_unstale":"2024-04-26T20:22:41.759460+0000","last_undegraded":"2024-04-26T20:22:41.759460+0000","last_fullsized":"2024-04-26T20:22:41.759460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:52.253630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.748484+0000","last_change":"2024-04-26T20:22:41.748484+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.748484+0000","last_undegraded":"2024-04-26T20:22:41.748484+0000","last_fullsized":"2024-04-26T20:22:41.748484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.765234+0000","last_change":"2024-04-26T20:22:41.760981+0000","last_active":"2024-04-26T20:22:41.765234+0000","last_peered":"2024-04-26T20:22:41.765234+0000","last_clean":"2024-04-26T20:22:41.765234+0000","last_became_active":"2024-04-26T20:22:41.760700+0000","last_became_peered":"2024-04-26T20:22:41.760700+0000","last_unstale":"2024-04-26T20:22:41.765234+0000","last_undegraded":"2024-04-26T20:22:41.765234+0000","last_fullsized":"2024-04-26T20:22:41.765234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:20.612907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:22:41.750327+0000","last_change":"2024-04-26T20:22:41.750327+0000","last_active":"2024-04-26T20:22:41.730889+0000","last_peered":"2024-04-26T20:22:41.730889+0000","last_clean":"2024-04-26T20:22:41.730889+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:22:41.750327+0000","last_undegraded":"2024-04-26T20:22:41.750327+0000","last_fullsized":"2024-04-26T20:22:41.750327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.760640+0000","last_change":"2024-04-26T20:22:41.755373+0000","last_active":"2024-04-26T20:22:41.760640+0000","last_peered":"2024-04-26T20:22:41.760640+0000","last_clean":"2024-04-26T20:22:41.760640+0000","last_became_active":"2024-04-26T20:22:41.755087+0000","last_became_peered":"2024-04-26T20:22:41.755087+0000","last_unstale":"2024-04-26T20:22:41.760640+0000","last_undegraded":"2024-04-26T20:22:41.760640+0000","last_fullsized":"2024-04-26T20:22:41.760640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:22:41.756741+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:22:41.756741+0000","last_peered":"2024-04-26T20:22:41.756741+0000","last_clean":"2024-04-26T20:22:41.756741+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:22:41.756741+0000","last_undegraded":"2024-04-26T20:22:41.756741+0000","last_fullsized":"2024-04-26T20:22:41.756741+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:22:49.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:22:49.437 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:22:51.218 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:22:51.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.330+0000 7ff7a415f640 1 Processor -- start 2024-04-26T20:22:51.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.331+0000 7ff7a415f640 1 -- start start 2024-04-26T20:22:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.331+0000 7ff7a415f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff79c0d3820 0x7ff79c0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.331+0000 7ff7a415f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79c0d0210 con 0x7ff79c162b40 2024-04-26T20:22:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.331+0000 7ff7a415f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79c0d04f0 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.331+0000 7ff7a415f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff79c0d41c0 con 0x7ff79c0d3820 2024-04-26T20:22:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.332+0000 7ff7a16fa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff79c0d3820 0x7ff79c0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.332+0000 7ff7a1efb640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff79c162b40 0x7ff79c0d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52612/0 (socket says 172.21.15.37:52612) 2024-04-26T20:22:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.332+0000 7ff7a1efb640 1 -- 172.21.15.37:0/2833076289 learned_addr learned my addr 172.21.15.37:0/2833076289 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.332+0000 7ff7a16fa640 1 -- 172.21.15.37:0/2833076289 >> v1:172.21.15.134:6789/0 conn(0x7ff79c0b4ba0 legacy=0x7ff79c0b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.332+0000 7ff7a16fa640 1 -- 172.21.15.37:0/2833076289 >> v1:172.21.15.37:6789/0 conn(0x7ff79c162b40 legacy=0x7ff79c0d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.332+0000 7ff7a16fa640 1 -- 172.21.15.37:0/2833076289 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79c0d07d0 con 0x7ff79c0d3820 2024-04-26T20:22:51.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.332+0000 7ff7a0ef9640 1 -- 172.21.15.37:0/2833076289 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3972291078 0 0) 0x7ff79c0d0210 con 0x7ff79c162b40 2024-04-26T20:22:51.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.333+0000 7ff7a16fa640 1 --2- 172.21.15.37:0/2833076289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff79c0d3820 0x7ff79c0d3c80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff7900097a0 tx=0x7ff79002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d92ab0f39eed161 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:51.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.333+0000 7ff7a0ef9640 1 -- 172.21.15.37:0/2833076289 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790037070 con 0x7ff79c0d3820 2024-04-26T20:22:51.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.333+0000 7ff7a0ef9640 1 -- 172.21.15.37:0/2833076289 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff790032440 con 0x7ff79c0d3820 2024-04-26T20:22:51.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.333+0000 7ff7a0ef9640 1 -- 172.21.15.37:0/2833076289 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790032810 con 0x7ff79c0d3820 2024-04-26T20:22:51.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.333+0000 7ff7a415f640 1 -- 172.21.15.37:0/2833076289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff79c0d3820 msgr2=0x7ff79c0d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.333+0000 7ff7a415f640 1 --2- 172.21.15.37:0/2833076289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff79c0d3820 0x7ff79c0d3c80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff7900097a0 tx=0x7ff79002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.333+0000 7ff7a415f640 1 -- 172.21.15.37:0/2833076289 shutdown_connections 2024-04-26T20:22:51.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.333+0000 7ff7a415f640 1 --2- 172.21.15.37:0/2833076289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff79c0d3820 0x7ff79c0d3c80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.333+0000 7ff7a415f640 1 -- 172.21.15.37:0/2833076289 >> 172.21.15.37:0/2833076289 conn(0x7ff79c09e550 msgr2=0x7ff79c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:51.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.334+0000 7ff7a415f640 1 -- 172.21.15.37:0/2833076289 shutdown_connections 2024-04-26T20:22:51.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.334+0000 7ff7a415f640 1 -- 172.21.15.37:0/2833076289 wait complete. 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.335+0000 7ff7a415f640 1 Processor -- start 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.335+0000 7ff7a415f640 1 -- start start 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.335+0000 7ff7a415f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff79c0b4ba0 0x7ff79c0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.335+0000 7ff7a415f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff79c0d3820 0x7ff79c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.335+0000 7ff7a415f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff79c162b40 0x7ff79c0b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.335+0000 7ff7a415f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c0d12b0 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.335+0000 7ff7a415f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c0d0d90 con 0x7ff79c0d3820 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.335+0000 7ff7a415f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff79c0cfe30 con 0x7ff79c162b40 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a1efb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff79c0b4ba0 0x7ff79c0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a1efb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff79c0b4ba0 0x7ff79c0cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35912/0 (socket says 172.21.15.37:35912) 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a1efb640 1 -- 172.21.15.37:0/3954152046 learned_addr learned my addr 172.21.15.37:0/3954152046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a16fa640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff79c0d3820 0x7ff79c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a26fc640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff79c162b40 0x7ff79c0b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a1efb640 1 -- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff79c162b40 msgr2=0x7ff79c0b86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a1efb640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff79c162b40 0x7ff79c0b86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a1efb640 1 -- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff79c0d3820 msgr2=0x7ff79c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a1efb640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff79c0d3820 0x7ff79c0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a1efb640 1 -- 172.21.15.37:0/3954152046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79c0ce400 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a26fc640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff79c162b40 0x7ff79c0b86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a16fa640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff79c0d3820 0x7ff79c0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a1efb640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff79c0b4ba0 0x7ff79c0cdb20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c4d0 tx=0x7ff79400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff79a7fc640 1 -- 172.21.15.37:0/3954152046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794018070 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff79a7fc640 1 -- 172.21.15.37:0/3954152046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff794007500 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff79a7fc640 1 -- 172.21.15.37:0/3954152046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794004a20 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a415f640 1 -- 172.21.15.37:0/3954152046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79c0ce6e0 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.336+0000 7ff7a415f640 1 -- 172.21.15.37:0/3954152046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff79c0cef80 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.337+0000 7ff7a415f640 1 -- 172.21.15.37:0/3954152046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff79c0ce8b0 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.338+0000 7ff79a7fc640 1 -- 172.21.15.37:0/3954152046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff79401b070 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.338+0000 7ff79a7fc640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff78806c8f0 0x7ff78806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.338+0000 7ff79a7fc640 1 -- 172.21.15.37:0/3954152046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff79408ae90 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.338+0000 7ff7a16fa640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff78806c8f0 0x7ff78806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.339+0000 7ff7a16fa640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff78806c8f0 0x7ff78806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff790034000 tx=0x7ff79002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.343+0000 7ff79a7fc640 1 -- 172.21.15.37:0/3954152046 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7940592c0 con 0x7ff79c0b4ba0 2024-04-26T20:22:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.528+0000 7ff7a415f640 1 -- 172.21.15.37:0/3954152046 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff79c162140 con 0x7ff78806c8f0 2024-04-26T20:22:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.534+0000 7ff79a7fc640 1 -- 172.21.15.37:0/3954152046 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251949 (secure 0 0 0) 0x7ff79c162140 con 0x7ff78806c8f0 2024-04-26T20:22:51.537 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:51.539 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:22:51.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 -- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff78806c8f0 msgr2=0x7ff78806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff78806c8f0 0x7ff78806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff790034000 tx=0x7ff79002e000 comp rx=0 tx=0).stop 2024-04-26T20:22:51.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 -- 172.21.15.37:0/3954152046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff79c0b4ba0 msgr2=0x7ff79c0cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff79c0b4ba0 0x7ff79c0cdb20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c4d0 tx=0x7ff79400c990 comp rx=0 tx=0).stop 2024-04-26T20:22:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 -- 172.21.15.37:0/3954152046 shutdown_connections 2024-04-26T20:22:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff78806c8f0 0x7ff78806edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff79c162b40 0x7ff79c0b86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff79c0d3820 0x7ff79c0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 --2- 172.21.15.37:0/3954152046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff79c0b4ba0 0x7ff79c0cdb20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 -- 172.21.15.37:0/3954152046 >> 172.21.15.37:0/3954152046 conn(0x7ff79c09e550 msgr2=0x7ff79c06db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 -- 172.21.15.37:0/3954152046 shutdown_connections 2024-04-26T20:22:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.537+0000 7ff7a415f640 1 -- 172.21.15.37:0/3954152046 wait complete. 2024-04-26T20:22:51.549 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-26T20:22:51.348019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220304,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754416,"statfs":{"total":773094113280,"available":772868521984,"internally_reserved":0,"allocated":4235264,"data_stored":2358484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.607659"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923069+0000","last_change":"2024-04-26T20:22:42.763602+0000","last_active":"2024-04-26T20:22:43.923069+0000","last_peered":"2024-04-26T20:22:43.923069+0000","last_clean":"2024-04-26T20:22:43.923069+0000","last_became_active":"2024-04-26T20:22:42.763332+0000","last_became_peered":"2024-04-26T20:22:42.763332+0000","last_unstale":"2024-04-26T20:22:43.923069+0000","last_undegraded":"2024-04-26T20:22:43.923069+0000","last_fullsized":"2024-04-26T20:22:43.923069+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:29.588389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922069+0000","last_change":"2024-04-26T20:22:42.763371+0000","last_active":"2024-04-26T20:22:43.922069+0000","last_peered":"2024-04-26T20:22:43.922069+0000","last_clean":"2024-04-26T20:22:43.922069+0000","last_became_active":"2024-04-26T20:22:42.763082+0000","last_became_peered":"2024-04-26T20:22:42.763082+0000","last_unstale":"2024-04-26T20:22:43.922069+0000","last_undegraded":"2024-04-26T20:22:43.922069+0000","last_fullsized":"2024-04-26T20:22:43.922069+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:28.305698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923440+0000","last_change":"2024-04-26T20:22:42.760332+0000","last_active":"2024-04-26T20:22:43.923440+0000","last_peered":"2024-04-26T20:22:43.923440+0000","last_clean":"2024-04-26T20:22:43.923440+0000","last_became_active":"2024-04-26T20:22:42.759954+0000","last_became_peered":"2024-04-26T20:22:42.759954+0000","last_unstale":"2024-04-26T20:22:43.923440+0000","last_undegraded":"2024-04-26T20:22:43.923440+0000","last_fullsized":"2024-04-26T20:22:43.923440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:17.308969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757211+0000","last_change":"2024-04-26T20:22:42.767022+0000","last_active":"2024-04-26T20:22:43.757211+0000","last_peered":"2024-04-26T20:22:43.757211+0000","last_clean":"2024-04-26T20:22:43.757211+0000","last_became_active":"2024-04-26T20:22:42.766519+0000","last_became_peered":"2024-04-26T20:22:42.766519+0000","last_unstale":"2024-04-26T20:22:43.757211+0000","last_undegraded":"2024-04-26T20:22:43.757211+0000","last_fullsized":"2024-04-26T20:22:43.757211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:52.002234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.756888+0000","last_change":"2024-04-26T20:22:42.764721+0000","last_active":"2024-04-26T20:22:43.756888+0000","last_peered":"2024-04-26T20:22:43.756888+0000","last_clean":"2024-04-26T20:22:43.756888+0000","last_became_active":"2024-04-26T20:22:42.764352+0000","last_became_peered":"2024-04-26T20:22:42.764352+0000","last_unstale":"2024-04-26T20:22:43.756888+0000","last_undegraded":"2024-04-26T20:22:43.756888+0000","last_fullsized":"2024-04-26T20:22:43.756888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:59.338426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.921868+0000","last_change":"2024-04-26T20:22:42.766412+0000","last_active":"2024-04-26T20:22:43.921868+0000","last_peered":"2024-04-26T20:22:43.921868+0000","last_clean":"2024-04-26T20:22:43.921868+0000","last_became_active":"2024-04-26T20:22:42.766135+0000","last_became_peered":"2024-04-26T20:22:42.766135+0000","last_unstale":"2024-04-26T20:22:43.921868+0000","last_undegraded":"2024-04-26T20:22:43.921868+0000","last_fullsized":"2024-04-26T20:22:43.921868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:17.308969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922601+0000","last_change":"2024-04-26T20:22:42.757752+0000","last_active":"2024-04-26T20:22:43.922601+0000","last_peered":"2024-04-26T20:22:43.922601+0000","last_clean":"2024-04-26T20:22:43.922601+0000","last_became_active":"2024-04-26T20:22:42.757468+0000","last_became_peered":"2024-04-26T20:22:42.757468+0000","last_unstale":"2024-04-26T20:22:43.922601+0000","last_undegraded":"2024-04-26T20:22:43.922601+0000","last_fullsized":"2024-04-26T20:22:43.922601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:39.200323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922520+0000","last_change":"2024-04-26T20:22:42.766435+0000","last_active":"2024-04-26T20:22:43.922520+0000","last_peered":"2024-04-26T20:22:43.922520+0000","last_clean":"2024-04-26T20:22:43.922520+0000","last_became_active":"2024-04-26T20:22:42.766141+0000","last_became_peered":"2024-04-26T20:22:42.766141+0000","last_unstale":"2024-04-26T20:22:43.922520+0000","last_undegraded":"2024-04-26T20:22:43.922520+0000","last_fullsized":"2024-04-26T20:22:43.922520+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:28.135478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924579+0000","last_change":"2024-04-26T20:22:42.769918+0000","last_active":"2024-04-26T20:22:43.924579+0000","last_peered":"2024-04-26T20:22:43.924579+0000","last_clean":"2024-04-26T20:22:43.924579+0000","last_became_active":"2024-04-26T20:22:42.769652+0000","last_became_peered":"2024-04-26T20:22:42.769652+0000","last_unstale":"2024-04-26T20:22:43.924579+0000","last_undegraded":"2024-04-26T20:22:43.924579+0000","last_fullsized":"2024-04-26T20:22:43.924579+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:52.555081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757379+0000","last_change":"2024-04-26T20:22:42.767558+0000","last_active":"2024-04-26T20:22:43.757379+0000","last_peered":"2024-04-26T20:22:43.757379+0000","last_clean":"2024-04-26T20:22:43.757379+0000","last_became_active":"2024-04-26T20:22:42.767255+0000","last_became_peered":"2024-04-26T20:22:42.767255+0000","last_unstale":"2024-04-26T20:22:43.757379+0000","last_undegraded":"2024-04-26T20:22:43.757379+0000","last_fullsized":"2024-04-26T20:22:43.757379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:17.560817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923199+0000","last_change":"2024-04-26T20:22:42.759315+0000","last_active":"2024-04-26T20:22:43.923199+0000","last_peered":"2024-04-26T20:22:43.923199+0000","last_clean":"2024-04-26T20:22:43.923199+0000","last_became_active":"2024-04-26T20:22:42.759026+0000","last_became_peered":"2024-04-26T20:22:42.759026+0000","last_unstale":"2024-04-26T20:22:43.923199+0000","last_undegraded":"2024-04-26T20:22:43.923199+0000","last_fullsized":"2024-04-26T20:22:43.923199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:31.160194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923395+0000","last_change":"2024-04-26T20:22:42.771476+0000","last_active":"2024-04-26T20:22:43.923395+0000","last_peered":"2024-04-26T20:22:43.923395+0000","last_clean":"2024-04-26T20:22:43.923395+0000","last_became_active":"2024-04-26T20:22:42.771232+0000","last_became_peered":"2024-04-26T20:22:42.771232+0000","last_unstale":"2024-04-26T20:22:43.923395+0000","last_undegraded":"2024-04-26T20:22:43.923395+0000","last_fullsized":"2024-04-26T20:22:43.923395+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:20.057280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757192+0000","last_change":"2024-04-26T20:22:42.759545+0000","last_active":"2024-04-26T20:22:43.757192+0000","last_peered":"2024-04-26T20:22:43.757192+0000","last_clean":"2024-04-26T20:22:43.757192+0000","last_became_active":"2024-04-26T20:22:42.759076+0000","last_became_peered":"2024-04-26T20:22:42.759076+0000","last_unstale":"2024-04-26T20:22:43.757192+0000","last_undegraded":"2024-04-26T20:22:43.757192+0000","last_fullsized":"2024-04-26T20:22:43.757192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:39.978430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.921954+0000","last_change":"2024-04-26T20:22:42.767141+0000","last_active":"2024-04-26T20:22:43.921954+0000","last_peered":"2024-04-26T20:22:43.921954+0000","last_clean":"2024-04-26T20:22:43.921954+0000","last_became_active":"2024-04-26T20:22:42.766778+0000","last_became_peered":"2024-04-26T20:22:42.766778+0000","last_unstale":"2024-04-26T20:22:43.921954+0000","last_undegraded":"2024-04-26T20:22:43.921954+0000","last_fullsized":"2024-04-26T20:22:43.921954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:34.916506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922725+0000","last_change":"2024-04-26T20:22:42.754523+0000","last_active":"2024-04-26T20:22:43.922725+0000","last_peered":"2024-04-26T20:22:43.922725+0000","last_clean":"2024-04-26T20:22:43.922725+0000","last_became_active":"2024-04-26T20:22:42.754224+0000","last_became_peered":"2024-04-26T20:22:42.754224+0000","last_unstale":"2024-04-26T20:22:43.922725+0000","last_undegraded":"2024-04-26T20:22:43.922725+0000","last_fullsized":"2024-04-26T20:22:43.922725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:17.560817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923026+0000","last_change":"2024-04-26T20:22:42.758716+0000","last_active":"2024-04-26T20:22:43.923026+0000","last_peered":"2024-04-26T20:22:43.923026+0000","last_clean":"2024-04-26T20:22:43.923026+0000","last_became_active":"2024-04-26T20:22:42.758449+0000","last_became_peered":"2024-04-26T20:22:42.758449+0000","last_unstale":"2024-04-26T20:22:43.923026+0000","last_undegraded":"2024-04-26T20:22:43.923026+0000","last_fullsized":"2024-04-26T20:22:43.923026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:14.917541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922542+0000","last_change":"2024-04-26T20:22:42.758144+0000","last_active":"2024-04-26T20:22:43.922542+0000","last_peered":"2024-04-26T20:22:43.922542+0000","last_clean":"2024-04-26T20:22:43.922542+0000","last_became_active":"2024-04-26T20:22:42.757834+0000","last_became_peered":"2024-04-26T20:22:42.757834+0000","last_unstale":"2024-04-26T20:22:43.922542+0000","last_undegraded":"2024-04-26T20:22:43.922542+0000","last_fullsized":"2024-04-26T20:22:43.922542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:34.916506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922112+0000","last_change":"2024-04-26T20:22:42.757480+0000","last_active":"2024-04-26T20:22:43.922112+0000","last_peered":"2024-04-26T20:22:43.922112+0000","last_clean":"2024-04-26T20:22:43.922112+0000","last_became_active":"2024-04-26T20:22:42.756640+0000","last_became_peered":"2024-04-26T20:22:42.756640+0000","last_unstale":"2024-04-26T20:22:43.922112+0000","last_undegraded":"2024-04-26T20:22:43.922112+0000","last_fullsized":"2024-04-26T20:22:43.922112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:37.447864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.921901+0000","last_change":"2024-04-26T20:22:42.760881+0000","last_active":"2024-04-26T20:22:43.921901+0000","last_peered":"2024-04-26T20:22:43.921901+0000","last_clean":"2024-04-26T20:22:43.921901+0000","last_became_active":"2024-04-26T20:22:42.760584+0000","last_became_peered":"2024-04-26T20:22:42.760584+0000","last_unstale":"2024-04-26T20:22:43.921901+0000","last_undegraded":"2024-04-26T20:22:43.921901+0000","last_fullsized":"2024-04-26T20:22:43.921901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:35.264524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757022+0000","last_change":"2024-04-26T20:22:42.765482+0000","last_active":"2024-04-26T20:22:43.757022+0000","last_peered":"2024-04-26T20:22:43.757022+0000","last_clean":"2024-04-26T20:22:43.757022+0000","last_became_active":"2024-04-26T20:22:42.765205+0000","last_became_peered":"2024-04-26T20:22:42.765205+0000","last_unstale":"2024-04-26T20:22:43.757022+0000","last_undegraded":"2024-04-26T20:22:43.757022+0000","last_fullsized":"2024-04-26T20:22:43.757022+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:52.795753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924048+0000","last_change":"2024-04-26T20:22:42.768793+0000","last_active":"2024-04-26T20:22:43.924048+0000","last_peered":"2024-04-26T20:22:43.924048+0000","last_clean":"2024-04-26T20:22:43.924048+0000","last_became_active":"2024-04-26T20:22:42.768597+0000","last_became_peered":"2024-04-26T20:22:42.768597+0000","last_unstale":"2024-04-26T20:22:43.924048+0000","last_undegraded":"2024-04-26T20:22:43.924048+0000","last_fullsized":"2024-04-26T20:22:43.924048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:29.588389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924179+0000","last_change":"2024-04-26T20:22:42.767349+0000","last_active":"2024-04-26T20:22:43.924179+0000","last_peered":"2024-04-26T20:22:43.924179+0000","last_clean":"2024-04-26T20:22:43.924179+0000","last_became_active":"2024-04-26T20:22:42.767007+0000","last_became_peered":"2024-04-26T20:22:42.767007+0000","last_unstale":"2024-04-26T20:22:43.924179+0000","last_undegraded":"2024-04-26T20:22:43.924179+0000","last_fullsized":"2024-04-26T20:22:43.924179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:54.523267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.752348+0000","last_change":"2024-04-26T20:22:42.751218+0000","last_active":"2024-04-26T20:22:43.752348+0000","last_peered":"2024-04-26T20:22:43.752348+0000","last_clean":"2024-04-26T20:22:43.752348+0000","last_became_active":"2024-04-26T20:22:42.750097+0000","last_became_peered":"2024-04-26T20:22:42.750097+0000","last_unstale":"2024-04-26T20:22:43.752348+0000","last_undegraded":"2024-04-26T20:22:43.752348+0000","last_fullsized":"2024-04-26T20:22:43.752348+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:29.588389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922965+0000","last_change":"2024-04-26T20:22:42.759445+0000","last_active":"2024-04-26T20:22:43.922965+0000","last_peered":"2024-04-26T20:22:43.922965+0000","last_clean":"2024-04-26T20:22:43.922965+0000","last_became_active":"2024-04-26T20:22:42.759172+0000","last_became_peered":"2024-04-26T20:22:42.759172+0000","last_unstale":"2024-04-26T20:22:43.922965+0000","last_undegraded":"2024-04-26T20:22:43.922965+0000","last_fullsized":"2024-04-26T20:22:43.922965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:16.294471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757337+0000","last_change":"2024-04-26T20:22:42.759937+0000","last_active":"2024-04-26T20:22:43.757337+0000","last_peered":"2024-04-26T20:22:43.757337+0000","last_clean":"2024-04-26T20:22:43.757337+0000","last_became_active":"2024-04-26T20:22:42.758830+0000","last_became_peered":"2024-04-26T20:22:42.758830+0000","last_unstale":"2024-04-26T20:22:43.757337+0000","last_undegraded":"2024-04-26T20:22:43.757337+0000","last_fullsized":"2024-04-26T20:22:43.757337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:14.796203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922211+0000","last_change":"2024-04-26T20:22:42.756856+0000","last_active":"2024-04-26T20:22:43.922211+0000","last_peered":"2024-04-26T20:22:43.922211+0000","last_clean":"2024-04-26T20:22:43.922211+0000","last_became_active":"2024-04-26T20:22:42.756555+0000","last_became_peered":"2024-04-26T20:22:42.756555+0000","last_unstale":"2024-04-26T20:22:43.922211+0000","last_undegraded":"2024-04-26T20:22:43.922211+0000","last_fullsized":"2024-04-26T20:22:43.922211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:14.917541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757657+0000","last_change":"2024-04-26T20:22:42.760010+0000","last_active":"2024-04-26T20:22:43.757657+0000","last_peered":"2024-04-26T20:22:43.757657+0000","last_clean":"2024-04-26T20:22:43.757657+0000","last_became_active":"2024-04-26T20:22:42.759733+0000","last_became_peered":"2024-04-26T20:22:42.759733+0000","last_unstale":"2024-04-26T20:22:43.757657+0000","last_undegraded":"2024-04-26T20:22:43.757657+0000","last_fullsized":"2024-04-26T20:22:43.757657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:13.394611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923486+0000","last_change":"2024-04-26T20:22:42.751209+0000","last_active":"2024-04-26T20:22:43.923486+0000","last_peered":"2024-04-26T20:22:43.923486+0000","last_clean":"2024-04-26T20:22:43.923486+0000","last_became_active":"2024-04-26T20:22:42.750942+0000","last_became_peered":"2024-04-26T20:22:42.750942+0000","last_unstale":"2024-04-26T20:22:43.923486+0000","last_undegraded":"2024-04-26T20:22:43.923486+0000","last_fullsized":"2024-04-26T20:22:43.923486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:28.135478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924151+0000","last_change":"2024-04-26T20:22:42.767427+0000","last_active":"2024-04-26T20:22:43.924151+0000","last_peered":"2024-04-26T20:22:43.924151+0000","last_clean":"2024-04-26T20:22:43.924151+0000","last_became_active":"2024-04-26T20:22:42.767086+0000","last_became_peered":"2024-04-26T20:22:42.767086+0000","last_unstale":"2024-04-26T20:22:43.924151+0000","last_undegraded":"2024-04-26T20:22:43.924151+0000","last_fullsized":"2024-04-26T20:22:43.924151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:24.386025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922347+0000","last_change":"2024-04-26T20:22:42.757110+0000","last_active":"2024-04-26T20:22:43.922347+0000","last_peered":"2024-04-26T20:22:43.922347+0000","last_clean":"2024-04-26T20:22:43.922347+0000","last_became_active":"2024-04-26T20:22:42.756814+0000","last_became_peered":"2024-04-26T20:22:42.756814+0000","last_unstale":"2024-04-26T20:22:43.922347+0000","last_undegraded":"2024-04-26T20:22:43.922347+0000","last_fullsized":"2024-04-26T20:22:43.922347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:17.308969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922464+0000","last_change":"2024-04-26T20:22:42.753440+0000","last_active":"2024-04-26T20:22:43.922464+0000","last_peered":"2024-04-26T20:22:43.922464+0000","last_clean":"2024-04-26T20:22:43.922464+0000","last_became_active":"2024-04-26T20:22:42.753177+0000","last_became_peered":"2024-04-26T20:22:42.753177+0000","last_unstale":"2024-04-26T20:22:43.922464+0000","last_undegraded":"2024-04-26T20:22:43.922464+0000","last_fullsized":"2024-04-26T20:22:43.922464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:52.399764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922692+0000","last_change":"2024-04-26T20:22:42.757475+0000","last_active":"2024-04-26T20:22:43.922692+0000","last_peered":"2024-04-26T20:22:43.922692+0000","last_clean":"2024-04-26T20:22:43.922692+0000","last_became_active":"2024-04-26T20:22:42.757164+0000","last_became_peered":"2024-04-26T20:22:42.757164+0000","last_unstale":"2024-04-26T20:22:43.922692+0000","last_undegraded":"2024-04-26T20:22:43.922692+0000","last_fullsized":"2024-04-26T20:22:43.922692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:28.305698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.752518+0000","last_change":"2024-04-26T20:22:42.750469+0000","last_active":"2024-04-26T20:22:43.752518+0000","last_peered":"2024-04-26T20:22:43.752518+0000","last_clean":"2024-04-26T20:22:43.752518+0000","last_became_active":"2024-04-26T20:22:42.750169+0000","last_became_peered":"2024-04-26T20:22:42.750169+0000","last_unstale":"2024-04-26T20:22:43.752518+0000","last_undegraded":"2024-04-26T20:22:43.752518+0000","last_fullsized":"2024-04-26T20:22:43.752518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:54.523267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923168+0000","last_change":"2024-04-26T20:22:41.756667+0000","last_active":"2024-04-26T20:22:43.923168+0000","last_peered":"2024-04-26T20:22:43.923168+0000","last_clean":"2024-04-26T20:22:43.923168+0000","last_became_active":"2024-04-26T20:22:41.756385+0000","last_became_peered":"2024-04-26T20:22:41.756385+0000","last_unstale":"2024-04-26T20:22:43.923168+0000","last_undegraded":"2024-04-26T20:22:43.923168+0000","last_fullsized":"2024-04-26T20:22:43.923168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:38.972810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923139+0000","last_change":"2024-04-26T20:22:42.760883+0000","last_active":"2024-04-26T20:22:43.923139+0000","last_peered":"2024-04-26T20:22:43.923139+0000","last_clean":"2024-04-26T20:22:43.923139+0000","last_became_active":"2024-04-26T20:22:42.760418+0000","last_became_peered":"2024-04-26T20:22:42.760418+0000","last_unstale":"2024-04-26T20:22:43.923139+0000","last_undegraded":"2024-04-26T20:22:43.923139+0000","last_fullsized":"2024-04-26T20:22:43.923139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:24.386025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.752539+0000","last_change":"2024-04-26T20:22:41.758415+0000","last_active":"2024-04-26T20:22:43.752539+0000","last_peered":"2024-04-26T20:22:43.752539+0000","last_clean":"2024-04-26T20:22:43.752539+0000","last_became_active":"2024-04-26T20:22:41.757712+0000","last_became_peered":"2024-04-26T20:22:41.757712+0000","last_unstale":"2024-04-26T20:22:43.752539+0000","last_undegraded":"2024-04-26T20:22:43.752539+0000","last_fullsized":"2024-04-26T20:22:43.752539+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:02.047370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924254+0000","last_change":"2024-04-26T20:22:42.769510+0000","last_active":"2024-04-26T20:22:43.924254+0000","last_peered":"2024-04-26T20:22:43.924254+0000","last_clean":"2024-04-26T20:22:43.924254+0000","last_became_active":"2024-04-26T20:22:42.769292+0000","last_became_peered":"2024-04-26T20:22:42.769292+0000","last_unstale":"2024-04-26T20:22:43.924254+0000","last_undegraded":"2024-04-26T20:22:43.924254+0000","last_fullsized":"2024-04-26T20:22:43.924254+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:08.381120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.921873+0000","last_change":"2024-04-26T20:22:41.759486+0000","last_active":"2024-04-26T20:22:43.921873+0000","last_peered":"2024-04-26T20:22:43.921873+0000","last_clean":"2024-04-26T20:22:43.921873+0000","last_became_active":"2024-04-26T20:22:41.759076+0000","last_became_peered":"2024-04-26T20:22:41.759076+0000","last_unstale":"2024-04-26T20:22:43.921873+0000","last_undegraded":"2024-04-26T20:22:43.921873+0000","last_fullsized":"2024-04-26T20:22:43.921873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:23.380405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757659+0000","last_change":"2024-04-26T20:22:42.760457+0000","last_active":"2024-04-26T20:22:43.757659+0000","last_peered":"2024-04-26T20:22:43.757659+0000","last_clean":"2024-04-26T20:22:43.757659+0000","last_became_active":"2024-04-26T20:22:42.759256+0000","last_became_peered":"2024-04-26T20:22:42.759256+0000","last_unstale":"2024-04-26T20:22:43.757659+0000","last_undegraded":"2024-04-26T20:22:43.757659+0000","last_fullsized":"2024-04-26T20:22:43.757659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:16.294471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757109+0000","last_change":"2024-04-26T20:22:41.754766+0000","last_active":"2024-04-26T20:22:43.757109+0000","last_peered":"2024-04-26T20:22:43.757109+0000","last_clean":"2024-04-26T20:22:43.757109+0000","last_became_active":"2024-04-26T20:22:41.754427+0000","last_became_peered":"2024-04-26T20:22:41.754427+0000","last_unstale":"2024-04-26T20:22:43.757109+0000","last_undegraded":"2024-04-26T20:22:43.757109+0000","last_fullsized":"2024-04-26T20:22:43.757109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:24.346362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922528+0000","last_change":"2024-04-26T20:22:42.760048+0000","last_active":"2024-04-26T20:22:43.922528+0000","last_peered":"2024-04-26T20:22:43.922528+0000","last_clean":"2024-04-26T20:22:43.922528+0000","last_became_active":"2024-04-26T20:22:42.759736+0000","last_became_peered":"2024-04-26T20:22:42.759736+0000","last_unstale":"2024-04-26T20:22:43.922528+0000","last_undegraded":"2024-04-26T20:22:43.922528+0000","last_fullsized":"2024-04-26T20:22:43.922528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:35.264524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924181+0000","last_change":"2024-04-26T20:22:41.759951+0000","last_active":"2024-04-26T20:22:43.924181+0000","last_peered":"2024-04-26T20:22:43.924181+0000","last_clean":"2024-04-26T20:22:43.924181+0000","last_became_active":"2024-04-26T20:22:41.759681+0000","last_became_peered":"2024-04-26T20:22:41.759681+0000","last_unstale":"2024-04-26T20:22:43.924181+0000","last_undegraded":"2024-04-26T20:22:43.924181+0000","last_fullsized":"2024-04-26T20:22:43.924181+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.332806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922872+0000","last_change":"2024-04-26T20:22:42.752570+0000","last_active":"2024-04-26T20:22:43.922872+0000","last_peered":"2024-04-26T20:22:43.922872+0000","last_clean":"2024-04-26T20:22:43.922872+0000","last_became_active":"2024-04-26T20:22:42.752294+0000","last_became_peered":"2024-04-26T20:22:42.752294+0000","last_unstale":"2024-04-26T20:22:43.922872+0000","last_undegraded":"2024-04-26T20:22:43.922872+0000","last_fullsized":"2024-04-26T20:22:43.922872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:00.359211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924417+0000","last_change":"2024-04-26T20:22:41.758461+0000","last_active":"2024-04-26T20:22:43.924417+0000","last_peered":"2024-04-26T20:22:43.924417+0000","last_clean":"2024-04-26T20:22:43.924417+0000","last_became_active":"2024-04-26T20:22:41.758166+0000","last_became_peered":"2024-04-26T20:22:41.758166+0000","last_unstale":"2024-04-26T20:22:43.924417+0000","last_undegraded":"2024-04-26T20:22:43.924417+0000","last_fullsized":"2024-04-26T20:22:43.924417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923855+0000","last_change":"2024-04-26T20:22:42.757098+0000","last_active":"2024-04-26T20:22:43.923855+0000","last_peered":"2024-04-26T20:22:43.923855+0000","last_clean":"2024-04-26T20:22:43.923855+0000","last_became_active":"2024-04-26T20:22:42.756823+0000","last_became_peered":"2024-04-26T20:22:42.756823+0000","last_unstale":"2024-04-26T20:22:43.923855+0000","last_undegraded":"2024-04-26T20:22:43.923855+0000","last_fullsized":"2024-04-26T20:22:43.923855+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:40.279503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.921674+0000","last_change":"2024-04-26T20:22:41.754578+0000","last_active":"2024-04-26T20:22:43.921674+0000","last_peered":"2024-04-26T20:22:43.921674+0000","last_clean":"2024-04-26T20:22:43.921674+0000","last_became_active":"2024-04-26T20:22:41.754286+0000","last_became_peered":"2024-04-26T20:22:41.754286+0000","last_unstale":"2024-04-26T20:22:43.921674+0000","last_undegraded":"2024-04-26T20:22:43.921674+0000","last_fullsized":"2024-04-26T20:22:43.921674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:51.394144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.756878+0000","last_change":"2024-04-26T20:22:42.762655+0000","last_active":"2024-04-26T20:22:43.756878+0000","last_peered":"2024-04-26T20:22:43.756878+0000","last_clean":"2024-04-26T20:22:43.756878+0000","last_became_active":"2024-04-26T20:22:42.762322+0000","last_became_peered":"2024-04-26T20:22:42.762322+0000","last_unstale":"2024-04-26T20:22:43.756878+0000","last_undegraded":"2024-04-26T20:22:43.756878+0000","last_fullsized":"2024-04-26T20:22:43.756878+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:35.914386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923804+0000","last_change":"2024-04-26T20:22:41.762053+0000","last_active":"2024-04-26T20:22:43.923804+0000","last_peered":"2024-04-26T20:22:43.923804+0000","last_clean":"2024-04-26T20:22:43.923804+0000","last_became_active":"2024-04-26T20:22:41.761753+0000","last_became_peered":"2024-04-26T20:22:41.761753+0000","last_unstale":"2024-04-26T20:22:43.923804+0000","last_undegraded":"2024-04-26T20:22:43.923804+0000","last_fullsized":"2024-04-26T20:22:43.923804+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:49.261035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923197+0000","last_change":"2024-04-26T20:22:42.762973+0000","last_active":"2024-04-26T20:22:43.923197+0000","last_peered":"2024-04-26T20:22:43.923197+0000","last_clean":"2024-04-26T20:22:43.923197+0000","last_became_active":"2024-04-26T20:22:42.762675+0000","last_became_peered":"2024-04-26T20:22:42.762675+0000","last_unstale":"2024-04-26T20:22:43.923197+0000","last_undegraded":"2024-04-26T20:22:43.923197+0000","last_fullsized":"2024-04-26T20:22:43.923197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:13.394611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922191+0000","last_change":"2024-04-26T20:22:41.761862+0000","last_active":"2024-04-26T20:22:43.922191+0000","last_peered":"2024-04-26T20:22:43.922191+0000","last_clean":"2024-04-26T20:22:43.922191+0000","last_became_active":"2024-04-26T20:22:41.761611+0000","last_became_peered":"2024-04-26T20:22:41.761611+0000","last_unstale":"2024-04-26T20:22:43.922191+0000","last_undegraded":"2024-04-26T20:22:43.922191+0000","last_fullsized":"2024-04-26T20:22:43.922191+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:53.517647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922074+0000","last_change":"2024-04-26T20:22:42.765715+0000","last_active":"2024-04-26T20:22:43.922074+0000","last_peered":"2024-04-26T20:22:43.922074+0000","last_clean":"2024-04-26T20:22:43.922074+0000","last_became_active":"2024-04-26T20:22:42.765417+0000","last_became_peered":"2024-04-26T20:22:42.765417+0000","last_unstale":"2024-04-26T20:22:43.922074+0000","last_undegraded":"2024-04-26T20:22:43.922074+0000","last_fullsized":"2024-04-26T20:22:43.922074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:29.472389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.752579+0000","last_change":"2024-04-26T20:22:41.757671+0000","last_active":"2024-04-26T20:22:43.752579+0000","last_peered":"2024-04-26T20:22:43.752579+0000","last_clean":"2024-04-26T20:22:43.752579+0000","last_became_active":"2024-04-26T20:22:41.757277+0000","last_became_peered":"2024-04-26T20:22:41.757277+0000","last_unstale":"2024-04-26T20:22:43.752579+0000","last_undegraded":"2024-04-26T20:22:43.752579+0000","last_fullsized":"2024-04-26T20:22:43.752579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:27.337131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922003+0000","last_change":"2024-04-26T20:22:42.759180+0000","last_active":"2024-04-26T20:22:43.922003+0000","last_peered":"2024-04-26T20:22:43.922003+0000","last_clean":"2024-04-26T20:22:43.922003+0000","last_became_active":"2024-04-26T20:22:42.757709+0000","last_became_peered":"2024-04-26T20:22:42.757709+0000","last_unstale":"2024-04-26T20:22:43.922003+0000","last_undegraded":"2024-04-26T20:22:43.922003+0000","last_fullsized":"2024-04-26T20:22:43.922003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:39.200323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923507+0000","last_change":"2024-04-26T20:22:41.760284+0000","last_active":"2024-04-26T20:22:43.923507+0000","last_peered":"2024-04-26T20:22:43.923507+0000","last_clean":"2024-04-26T20:22:43.923507+0000","last_became_active":"2024-04-26T20:22:41.759981+0000","last_became_peered":"2024-04-26T20:22:41.759981+0000","last_unstale":"2024-04-26T20:22:43.923507+0000","last_undegraded":"2024-04-26T20:22:43.923507+0000","last_fullsized":"2024-04-26T20:22:43.923507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:51.394144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923396+0000","last_change":"2024-04-26T20:22:42.761302+0000","last_active":"2024-04-26T20:22:43.923396+0000","last_peered":"2024-04-26T20:22:43.923396+0000","last_clean":"2024-04-26T20:22:43.923396+0000","last_became_active":"2024-04-26T20:22:42.760957+0000","last_became_peered":"2024-04-26T20:22:42.760957+0000","last_unstale":"2024-04-26T20:22:43.923396+0000","last_undegraded":"2024-04-26T20:22:43.923396+0000","last_fullsized":"2024-04-26T20:22:43.923396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:08.894957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922068+0000","last_change":"2024-04-26T20:22:41.759409+0000","last_active":"2024-04-26T20:22:43.922068+0000","last_peered":"2024-04-26T20:22:43.922068+0000","last_clean":"2024-04-26T20:22:43.922068+0000","last_became_active":"2024-04-26T20:22:41.758974+0000","last_became_peered":"2024-04-26T20:22:41.758974+0000","last_unstale":"2024-04-26T20:22:43.922068+0000","last_undegraded":"2024-04-26T20:22:43.922068+0000","last_fullsized":"2024-04-26T20:22:43.922068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:39.273883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757140+0000","last_change":"2024-04-26T20:22:42.762965+0000","last_active":"2024-04-26T20:22:43.757140+0000","last_peered":"2024-04-26T20:22:43.757140+0000","last_clean":"2024-04-26T20:22:43.757140+0000","last_became_active":"2024-04-26T20:22:42.762655+0000","last_became_peered":"2024-04-26T20:22:42.762655+0000","last_unstale":"2024-04-26T20:22:43.757140+0000","last_undegraded":"2024-04-26T20:22:43.757140+0000","last_fullsized":"2024-04-26T20:22:43.757140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:52.555081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924325+0000","last_change":"2024-04-26T20:22:41.760955+0000","last_active":"2024-04-26T20:22:43.924325+0000","last_peered":"2024-04-26T20:22:43.924325+0000","last_clean":"2024-04-26T20:22:43.924325+0000","last_became_active":"2024-04-26T20:22:41.760658+0000","last_became_peered":"2024-04-26T20:22:41.760658+0000","last_unstale":"2024-04-26T20:22:43.924325+0000","last_undegraded":"2024-04-26T20:22:43.924325+0000","last_fullsized":"2024-04-26T20:22:43.924325+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:33.766443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.921892+0000","last_change":"2024-04-26T20:22:42.763741+0000","last_active":"2024-04-26T20:22:43.921892+0000","last_peered":"2024-04-26T20:22:43.921892+0000","last_clean":"2024-04-26T20:22:43.921892+0000","last_became_active":"2024-04-26T20:22:42.763480+0000","last_became_peered":"2024-04-26T20:22:42.763480+0000","last_unstale":"2024-04-26T20:22:43.921892+0000","last_undegraded":"2024-04-26T20:22:43.921892+0000","last_fullsized":"2024-04-26T20:22:43.921892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:28.135478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757234+0000","last_change":"2024-04-26T20:22:41.751412+0000","last_active":"2024-04-26T20:22:43.757234+0000","last_peered":"2024-04-26T20:22:43.757234+0000","last_clean":"2024-04-26T20:22:43.757234+0000","last_became_active":"2024-04-26T20:22:41.751102+0000","last_became_peered":"2024-04-26T20:22:41.751102+0000","last_unstale":"2024-04-26T20:22:43.757234+0000","last_undegraded":"2024-04-26T20:22:43.757234+0000","last_fullsized":"2024-04-26T20:22:43.757234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:42.413094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922865+0000","last_change":"2024-04-26T20:22:42.757908+0000","last_active":"2024-04-26T20:22:43.922865+0000","last_peered":"2024-04-26T20:22:43.922865+0000","last_clean":"2024-04-26T20:22:43.922865+0000","last_became_active":"2024-04-26T20:22:42.757612+0000","last_became_peered":"2024-04-26T20:22:42.757612+0000","last_unstale":"2024-04-26T20:22:43.922865+0000","last_undegraded":"2024-04-26T20:22:43.922865+0000","last_fullsized":"2024-04-26T20:22:43.922865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:37.447864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.921603+0000","last_change":"2024-04-26T20:22:41.760709+0000","last_active":"2024-04-26T20:22:43.921603+0000","last_peered":"2024-04-26T20:22:43.921603+0000","last_clean":"2024-04-26T20:22:43.921603+0000","last_became_active":"2024-04-26T20:22:41.760281+0000","last_became_peered":"2024-04-26T20:22:41.760281+0000","last_unstale":"2024-04-26T20:22:43.921603+0000","last_undegraded":"2024-04-26T20:22:43.921603+0000","last_fullsized":"2024-04-26T20:22:43.921603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:35.230803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922484+0000","last_change":"2024-04-26T20:22:42.758716+0000","last_active":"2024-04-26T20:22:43.922484+0000","last_peered":"2024-04-26T20:22:43.922484+0000","last_clean":"2024-04-26T20:22:43.922484+0000","last_became_active":"2024-04-26T20:22:42.758449+0000","last_became_peered":"2024-04-26T20:22:42.758449+0000","last_unstale":"2024-04-26T20:22:43.922484+0000","last_undegraded":"2024-04-26T20:22:43.922484+0000","last_fullsized":"2024-04-26T20:22:43.922484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:40.279503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.921778+0000","last_change":"2024-04-26T20:22:41.759620+0000","last_active":"2024-04-26T20:22:43.921778+0000","last_peered":"2024-04-26T20:22:43.921778+0000","last_clean":"2024-04-26T20:22:43.921778+0000","last_became_active":"2024-04-26T20:22:41.759154+0000","last_became_peered":"2024-04-26T20:22:41.759154+0000","last_unstale":"2024-04-26T20:22:43.921778+0000","last_undegraded":"2024-04-26T20:22:43.921778+0000","last_fullsized":"2024-04-26T20:22:43.921778+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:48.639316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924357+0000","last_change":"2024-04-26T20:22:42.765744+0000","last_active":"2024-04-26T20:22:43.924357+0000","last_peered":"2024-04-26T20:22:43.924357+0000","last_clean":"2024-04-26T20:22:43.924357+0000","last_became_active":"2024-04-26T20:22:42.765461+0000","last_became_peered":"2024-04-26T20:22:42.765461+0000","last_unstale":"2024-04-26T20:22:43.924357+0000","last_undegraded":"2024-04-26T20:22:43.924357+0000","last_fullsized":"2024-04-26T20:22:43.924357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:35.914386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757463+0000","last_change":"2024-04-26T20:22:41.758134+0000","last_active":"2024-04-26T20:22:43.757463+0000","last_peered":"2024-04-26T20:22:43.757463+0000","last_clean":"2024-04-26T20:22:43.757463+0000","last_became_active":"2024-04-26T20:22:41.757853+0000","last_became_peered":"2024-04-26T20:22:41.757853+0000","last_unstale":"2024-04-26T20:22:43.757463+0000","last_undegraded":"2024-04-26T20:22:43.757463+0000","last_fullsized":"2024-04-26T20:22:43.757463+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:20.612907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923635+0000","last_change":"2024-04-26T20:22:42.760543+0000","last_active":"2024-04-26T20:22:43.923635+0000","last_peered":"2024-04-26T20:22:43.923635+0000","last_clean":"2024-04-26T20:22:43.923635+0000","last_became_active":"2024-04-26T20:22:42.760261+0000","last_became_peered":"2024-04-26T20:22:42.760261+0000","last_unstale":"2024-04-26T20:22:43.923635+0000","last_undegraded":"2024-04-26T20:22:43.923635+0000","last_fullsized":"2024-04-26T20:22:43.923635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:35.264524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924338+0000","last_change":"2024-04-26T20:22:41.756567+0000","last_active":"2024-04-26T20:22:43.924338+0000","last_peered":"2024-04-26T20:22:43.924338+0000","last_clean":"2024-04-26T20:22:43.924338+0000","last_became_active":"2024-04-26T20:22:41.756259+0000","last_became_peered":"2024-04-26T20:22:41.756259+0000","last_unstale":"2024-04-26T20:22:43.924338+0000","last_undegraded":"2024-04-26T20:22:43.924338+0000","last_fullsized":"2024-04-26T20:22:43.924338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:44.488098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757377+0000","last_change":"2024-04-26T20:22:42.766788+0000","last_active":"2024-04-26T20:22:43.757377+0000","last_peered":"2024-04-26T20:22:43.757377+0000","last_clean":"2024-04-26T20:22:43.757377+0000","last_became_active":"2024-04-26T20:22:42.766172+0000","last_became_peered":"2024-04-26T20:22:42.766172+0000","last_unstale":"2024-04-26T20:22:43.757377+0000","last_undegraded":"2024-04-26T20:22:43.757377+0000","last_fullsized":"2024-04-26T20:22:43.757377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:00.359211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924340+0000","last_change":"2024-04-26T20:22:41.757636+0000","last_active":"2024-04-26T20:22:43.924340+0000","last_peered":"2024-04-26T20:22:43.924340+0000","last_clean":"2024-04-26T20:22:43.924340+0000","last_became_active":"2024-04-26T20:22:41.757329+0000","last_became_peered":"2024-04-26T20:22:41.757329+0000","last_unstale":"2024-04-26T20:22:43.924340+0000","last_undegraded":"2024-04-26T20:22:43.924340+0000","last_fullsized":"2024-04-26T20:22:43.924340+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:44.639627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757289+0000","last_change":"2024-04-26T20:22:42.759418+0000","last_active":"2024-04-26T20:22:43.757289+0000","last_peered":"2024-04-26T20:22:43.757289+0000","last_clean":"2024-04-26T20:22:43.757289+0000","last_became_active":"2024-04-26T20:22:42.759115+0000","last_became_peered":"2024-04-26T20:22:42.759115+0000","last_unstale":"2024-04-26T20:22:43.757289+0000","last_undegraded":"2024-04-26T20:22:43.757289+0000","last_fullsized":"2024-04-26T20:22:43.757289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:28.342751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.924341+0000","last_change":"2024-04-26T20:22:41.759343+0000","last_active":"2024-04-26T20:22:43.924341+0000","last_peered":"2024-04-26T20:22:43.924341+0000","last_clean":"2024-04-26T20:22:43.924341+0000","last_became_active":"2024-04-26T20:22:41.758924+0000","last_became_peered":"2024-04-26T20:22:41.758924+0000","last_unstale":"2024-04-26T20:22:43.924341+0000","last_undegraded":"2024-04-26T20:22:43.924341+0000","last_fullsized":"2024-04-26T20:22:43.924341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:59.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.756859+0000","last_change":"2024-04-26T20:22:42.758580+0000","last_active":"2024-04-26T20:22:43.756859+0000","last_peered":"2024-04-26T20:22:43.756859+0000","last_clean":"2024-04-26T20:22:43.756859+0000","last_became_active":"2024-04-26T20:22:42.758320+0000","last_became_peered":"2024-04-26T20:22:42.758320+0000","last_unstale":"2024-04-26T20:22:43.756859+0000","last_undegraded":"2024-04-26T20:22:43.756859+0000","last_fullsized":"2024-04-26T20:22:43.756859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:56.938112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.752501+0000","last_change":"2024-04-26T20:22:41.747174+0000","last_active":"2024-04-26T20:22:43.752501+0000","last_peered":"2024-04-26T20:22:43.752501+0000","last_clean":"2024-04-26T20:22:43.752501+0000","last_became_active":"2024-04-26T20:22:41.746153+0000","last_became_peered":"2024-04-26T20:22:41.746153+0000","last_unstale":"2024-04-26T20:22:43.752501+0000","last_undegraded":"2024-04-26T20:22:43.752501+0000","last_fullsized":"2024-04-26T20:22:43.752501+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:42.413094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757073+0000","last_change":"2024-04-26T20:22:42.755105+0000","last_active":"2024-04-26T20:22:43.757073+0000","last_peered":"2024-04-26T20:22:43.757073+0000","last_clean":"2024-04-26T20:22:43.757073+0000","last_became_active":"2024-04-26T20:22:42.754842+0000","last_became_peered":"2024-04-26T20:22:42.754842+0000","last_unstale":"2024-04-26T20:22:43.757073+0000","last_undegraded":"2024-04-26T20:22:43.757073+0000","last_fullsized":"2024-04-26T20:22:43.757073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:36.286791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922783+0000","last_change":"2024-04-26T20:22:41.755144+0000","last_active":"2024-04-26T20:22:43.922783+0000","last_peered":"2024-04-26T20:22:43.922783+0000","last_clean":"2024-04-26T20:22:43.922783+0000","last_became_active":"2024-04-26T20:22:41.754847+0000","last_became_peered":"2024-04-26T20:22:41.754847+0000","last_unstale":"2024-04-26T20:22:43.922783+0000","last_undegraded":"2024-04-26T20:22:43.922783+0000","last_fullsized":"2024-04-26T20:22:43.922783+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:27.337131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.756732+0000","last_change":"2024-04-26T20:22:42.757781+0000","last_active":"2024-04-26T20:22:43.756732+0000","last_peered":"2024-04-26T20:22:43.756732+0000","last_clean":"2024-04-26T20:22:43.756732+0000","last_became_active":"2024-04-26T20:22:42.757514+0000","last_became_peered":"2024-04-26T20:22:42.757514+0000","last_unstale":"2024-04-26T20:22:43.756732+0000","last_undegraded":"2024-04-26T20:22:43.756732+0000","last_fullsized":"2024-04-26T20:22:43.756732+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:00.971430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923908+0000","last_change":"2024-04-26T20:22:41.759321+0000","last_active":"2024-04-26T20:22:43.923908+0000","last_peered":"2024-04-26T20:22:43.923908+0000","last_clean":"2024-04-26T20:22:43.923908+0000","last_became_active":"2024-04-26T20:22:41.759034+0000","last_became_peered":"2024-04-26T20:22:41.759034+0000","last_unstale":"2024-04-26T20:22:43.923908+0000","last_undegraded":"2024-04-26T20:22:43.923908+0000","last_fullsized":"2024-04-26T20:22:43.923908+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:55.932492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922334+0000","last_change":"2024-04-26T20:22:42.757172+0000","last_active":"2024-04-26T20:22:43.922334+0000","last_peered":"2024-04-26T20:22:43.922334+0000","last_clean":"2024-04-26T20:22:43.922334+0000","last_became_active":"2024-04-26T20:22:42.756871+0000","last_became_peered":"2024-04-26T20:22:42.756871+0000","last_unstale":"2024-04-26T20:22:43.922334+0000","last_undegraded":"2024-04-26T20:22:43.922334+0000","last_fullsized":"2024-04-26T20:22:43.922334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:20.057280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922078+0000","last_change":"2024-04-26T20:22:41.760401+0000","last_active":"2024-04-26T20:22:43.922078+0000","last_peered":"2024-04-26T20:22:43.922078+0000","last_clean":"2024-04-26T20:22:43.922078+0000","last_became_active":"2024-04-26T20:22:41.760080+0000","last_became_peered":"2024-04-26T20:22:41.760080+0000","last_unstale":"2024-04-26T20:22:43.922078+0000","last_undegraded":"2024-04-26T20:22:43.922078+0000","last_fullsized":"2024-04-26T20:22:43.922078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:35.230803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757037+0000","last_change":"2024-04-26T20:22:42.762089+0000","last_active":"2024-04-26T20:22:43.757037+0000","last_peered":"2024-04-26T20:22:43.757037+0000","last_clean":"2024-04-26T20:22:43.757037+0000","last_became_active":"2024-04-26T20:22:42.761829+0000","last_became_peered":"2024-04-26T20:22:42.761829+0000","last_unstale":"2024-04-26T20:22:43.757037+0000","last_undegraded":"2024-04-26T20:22:43.757037+0000","last_fullsized":"2024-04-26T20:22:43.757037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:34.772063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757528+0000","last_change":"2024-04-26T20:22:41.756976+0000","last_active":"2024-04-26T20:22:43.757528+0000","last_peered":"2024-04-26T20:22:43.757528+0000","last_clean":"2024-04-26T20:22:43.757528+0000","last_became_active":"2024-04-26T20:22:41.756639+0000","last_became_peered":"2024-04-26T20:22:41.756639+0000","last_unstale":"2024-04-26T20:22:43.757528+0000","last_undegraded":"2024-04-26T20:22:43.757528+0000","last_fullsized":"2024-04-26T20:22:43.757528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:02.047370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923691+0000","last_change":"2024-04-26T20:22:42.764193+0000","last_active":"2024-04-26T20:22:43.923691+0000","last_peered":"2024-04-26T20:22:43.923691+0000","last_clean":"2024-04-26T20:22:43.923691+0000","last_became_active":"2024-04-26T20:22:42.762858+0000","last_became_peered":"2024-04-26T20:22:42.762858+0000","last_unstale":"2024-04-26T20:22:43.923691+0000","last_undegraded":"2024-04-26T20:22:43.923691+0000","last_fullsized":"2024-04-26T20:22:43.923691+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:34.916506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.752665+0000","last_change":"2024-04-26T20:22:41.747756+0000","last_active":"2024-04-26T20:22:43.752665+0000","last_peered":"2024-04-26T20:22:43.752665+0000","last_clean":"2024-04-26T20:22:43.752665+0000","last_became_active":"2024-04-26T20:22:41.747460+0000","last_became_peered":"2024-04-26T20:22:41.747460+0000","last_unstale":"2024-04-26T20:22:43.752665+0000","last_undegraded":"2024-04-26T20:22:43.752665+0000","last_fullsized":"2024-04-26T20:22:43.752665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:20.522370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.752445+0000","last_change":"2024-04-26T20:22:42.758175+0000","last_active":"2024-04-26T20:22:43.752445+0000","last_peered":"2024-04-26T20:22:43.752445+0000","last_clean":"2024-04-26T20:22:43.752445+0000","last_became_active":"2024-04-26T20:22:42.757873+0000","last_became_peered":"2024-04-26T20:22:42.757873+0000","last_unstale":"2024-04-26T20:22:43.752445+0000","last_undegraded":"2024-04-26T20:22:43.752445+0000","last_fullsized":"2024-04-26T20:22:43.752445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:24.386025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757470+0000","last_change":"2024-04-26T20:22:41.750670+0000","last_active":"2024-04-26T20:22:43.757470+0000","last_peered":"2024-04-26T20:22:43.757470+0000","last_clean":"2024-04-26T20:22:43.757470+0000","last_became_active":"2024-04-26T20:22:41.750317+0000","last_became_peered":"2024-04-26T20:22:41.750317+0000","last_unstale":"2024-04-26T20:22:43.757470+0000","last_undegraded":"2024-04-26T20:22:43.757470+0000","last_fullsized":"2024-04-26T20:22:43.757470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:27.337131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922724+0000","last_change":"2024-04-26T20:22:42.759526+0000","last_active":"2024-04-26T20:22:43.922724+0000","last_peered":"2024-04-26T20:22:43.922724+0000","last_clean":"2024-04-26T20:22:43.922724+0000","last_became_active":"2024-04-26T20:22:42.759256+0000","last_became_peered":"2024-04-26T20:22:42.759256+0000","last_unstale":"2024-04-26T20:22:43.922724+0000","last_undegraded":"2024-04-26T20:22:43.922724+0000","last_fullsized":"2024-04-26T20:22:43.922724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:49.644936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.752609+0000","last_change":"2024-04-26T20:22:41.754948+0000","last_active":"2024-04-26T20:22:43.752609+0000","last_peered":"2024-04-26T20:22:43.752609+0000","last_clean":"2024-04-26T20:22:43.752609+0000","last_became_active":"2024-04-26T20:22:41.754664+0000","last_became_peered":"2024-04-26T20:22:41.754664+0000","last_unstale":"2024-04-26T20:22:43.752609+0000","last_undegraded":"2024-04-26T20:22:43.752609+0000","last_fullsized":"2024-04-26T20:22:43.752609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:20.612907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.752573+0000","last_change":"2024-04-26T20:22:42.753257+0000","last_active":"2024-04-26T20:22:43.752573+0000","last_peered":"2024-04-26T20:22:43.752573+0000","last_clean":"2024-04-26T20:22:43.752573+0000","last_became_active":"2024-04-26T20:22:42.752976+0000","last_became_peered":"2024-04-26T20:22:42.752976+0000","last_unstale":"2024-04-26T20:22:43.752573+0000","last_undegraded":"2024-04-26T20:22:43.752573+0000","last_fullsized":"2024-04-26T20:22:43.752573+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:08.381120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.757305+0000","last_change":"2024-04-26T20:22:41.756879+0000","last_active":"2024-04-26T20:22:43.757305+0000","last_peered":"2024-04-26T20:22:43.757305+0000","last_clean":"2024-04-26T20:22:43.757305+0000","last_became_active":"2024-04-26T20:22:41.756600+0000","last_became_peered":"2024-04-26T20:22:41.756600+0000","last_unstale":"2024-04-26T20:22:43.757305+0000","last_undegraded":"2024-04-26T20:22:43.757305+0000","last_fullsized":"2024-04-26T20:22:43.757305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:20.522370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.921633+0000","last_change":"2024-04-26T20:22:42.756178+0000","last_active":"2024-04-26T20:22:43.921633+0000","last_peered":"2024-04-26T20:22:43.921633+0000","last_clean":"2024-04-26T20:22:43.921633+0000","last_became_active":"2024-04-26T20:22:42.755737+0000","last_became_peered":"2024-04-26T20:22:42.755737+0000","last_unstale":"2024-04-26T20:22:43.921633+0000","last_undegraded":"2024-04-26T20:22:43.921633+0000","last_fullsized":"2024-04-26T20:22:43.921633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:20.057280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923995+0000","last_change":"2024-04-26T20:22:41.756827+0000","last_active":"2024-04-26T20:22:43.923995+0000","last_peered":"2024-04-26T20:22:43.923995+0000","last_clean":"2024-04-26T20:22:43.923995+0000","last_became_active":"2024-04-26T20:22:41.756542+0000","last_became_peered":"2024-04-26T20:22:41.756542+0000","last_unstale":"2024-04-26T20:22:43.923995+0000","last_undegraded":"2024-04-26T20:22:43.923995+0000","last_fullsized":"2024-04-26T20:22:43.923995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:51.790133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923875+0000","last_change":"2024-04-26T20:22:42.767623+0000","last_active":"2024-04-26T20:22:43.923875+0000","last_peered":"2024-04-26T20:22:43.923875+0000","last_clean":"2024-04-26T20:22:43.923875+0000","last_became_active":"2024-04-26T20:22:42.767100+0000","last_became_peered":"2024-04-26T20:22:42.767100+0000","last_unstale":"2024-04-26T20:22:43.923875+0000","last_undegraded":"2024-04-26T20:22:43.923875+0000","last_fullsized":"2024-04-26T20:22:43.923875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:49.644936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.921630+0000","last_change":"2024-04-26T20:22:41.760981+0000","last_active":"2024-04-26T20:22:43.921630+0000","last_peered":"2024-04-26T20:22:43.921630+0000","last_clean":"2024-04-26T20:22:43.921630+0000","last_became_active":"2024-04-26T20:22:41.760700+0000","last_became_peered":"2024-04-26T20:22:41.760700+0000","last_unstale":"2024-04-26T20:22:43.921630+0000","last_undegraded":"2024-04-26T20:22:43.921630+0000","last_fullsized":"2024-04-26T20:22:43.921630+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:28.582769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922826+0000","last_change":"2024-04-26T20:22:42.755302+0000","last_active":"2024-04-26T20:22:43.922826+0000","last_peered":"2024-04-26T20:22:43.922826+0000","last_clean":"2024-04-26T20:22:43.922826+0000","last_became_active":"2024-04-26T20:22:42.755033+0000","last_became_peered":"2024-04-26T20:22:42.755033+0000","last_unstale":"2024-04-26T20:22:43.922826+0000","last_undegraded":"2024-04-26T20:22:43.922826+0000","last_fullsized":"2024-04-26T20:22:43.922826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:41.730889+0000","last_clean_scrub_stamp":"2024-04-26T20:22:41.730889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:29.472389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.922458+0000","last_change":"2024-04-26T20:22:41.755373+0000","last_active":"2024-04-26T20:22:43.922458+0000","last_peered":"2024-04-26T20:22:43.922458+0000","last_clean":"2024-04-26T20:22:43.922458+0000","last_became_active":"2024-04-26T20:22:41.755087+0000","last_became_peered":"2024-04-26T20:22:41.755087+0000","last_unstale":"2024-04-26T20:22:43.922458+0000","last_undegraded":"2024-04-26T20:22:43.922458+0000","last_fullsized":"2024-04-26T20:22:43.922458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:22:40.725268+0000","last_clean_scrub_stamp":"2024-04-26T20:22:40.725268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:35.281171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":194,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:22:43.923120+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:22:43.923120+0000","last_peered":"2024-04-26T20:22:43.923120+0000","last_clean":"2024-04-26T20:22:43.923120+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:22:43.923120+0000","last_undegraded":"2024-04-26T20:22:43.923120+0000","last_fullsized":"2024-04-26T20:22:43.923120+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:22:51.553 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:22:51.554 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T20:22:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.697+0000 7f3f51e83640 1 Processor -- start 2024-04-26T20:22:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.697+0000 7f3f51e83640 1 -- start start 2024-04-26T20:22:51.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.697+0000 7f3f51e83640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3f4c0d2320 0x7f3f4c0d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:51.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.697+0000 7f3f51e83640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4c161510 con 0x7f3f4c1610c0 2024-04-26T20:22:51.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.697+0000 7f3f51e83640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4c0d0930 con 0x7f3f4c160800 2024-04-26T20:22:51.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.697+0000 7f3f51e83640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3f4c0d0c00 con 0x7f3f4c0d2320 2024-04-26T20:22:51.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.698+0000 7f3f4bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3f4c0d2320 0x7f3f4c0d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.698+0000 7f3f4affd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3f4c1610c0 0x7f3f4c0d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52614/0 (socket says 172.21.15.37:52614) 2024-04-26T20:22:51.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.698+0000 7f3f4affd640 1 -- 172.21.15.37:0/2610263804 learned_addr learned my addr 172.21.15.37:0/2610263804 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:51.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.698+0000 7f3f4bfff640 1 -- 172.21.15.37:0/2610263804 >> v1:172.21.15.134:6789/0 conn(0x7f3f4c160800 legacy=0x7f3f4c0d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:51.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.698+0000 7f3f4bfff640 1 -- 172.21.15.37:0/2610263804 >> v1:172.21.15.37:6789/0 conn(0x7f3f4c1610c0 legacy=0x7f3f4c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.698+0000 7f3f4a7fc640 1 -- 172.21.15.37:0/2610263804 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1451780567 0 0) 0x7f3f4c161510 con 0x7f3f4c1610c0 2024-04-26T20:22:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.698+0000 7f3f4bfff640 1 -- 172.21.15.37:0/2610263804 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c0d0ee0 con 0x7f3f4c0d2320 2024-04-26T20:22:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.699+0000 7f3f4bfff640 1 --2- 172.21.15.37:0/2610263804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c0d2320 0x7f3f4c0d3610 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009d90 tx=0x7f3f4402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860bd473cd24d8ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.699+0000 7f3f4a7fc640 1 -- 172.21.15.37:0/2610263804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44007740 con 0x7f3f4c0d2320 2024-04-26T20:22:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.699+0000 7f3f4a7fc640 1 -- 172.21.15.37:0/2610263804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f440078e0 con 0x7f3f4c0d2320 2024-04-26T20:22:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.699+0000 7f3f4a7fc640 1 -- 172.21.15.37:0/2610263804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44007cb0 con 0x7f3f4c0d2320 2024-04-26T20:22:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.699+0000 7f3f51e83640 1 -- 172.21.15.37:0/2610263804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c0d2320 msgr2=0x7f3f4c0d3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.699+0000 7f3f51e83640 1 --2- 172.21.15.37:0/2610263804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c0d2320 0x7f3f4c0d3610 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009d90 tx=0x7f3f4402a660 comp rx=0 tx=0).stop 2024-04-26T20:22:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.699+0000 7f3f51e83640 1 -- 172.21.15.37:0/2610263804 shutdown_connections 2024-04-26T20:22:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.699+0000 7f3f51e83640 1 --2- 172.21.15.37:0/2610263804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c0d2320 0x7f3f4c0d3610 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.699+0000 7f3f51e83640 1 -- 172.21.15.37:0/2610263804 >> 172.21.15.37:0/2610263804 conn(0x7f3f4c09e550 msgr2=0x7f3f4c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.699+0000 7f3f51e83640 1 -- 172.21.15.37:0/2610263804 shutdown_connections 2024-04-26T20:22:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.699+0000 7f3f51e83640 1 -- 172.21.15.37:0/2610263804 wait complete. 2024-04-26T20:22:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.701+0000 7f3f51e83640 1 Processor -- start 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.701+0000 7f3f51e83640 1 -- start start 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.701+0000 7f3f51e83640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f4c0d2320 0x7f3f4c1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.701+0000 7f3f51e83640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f4c160800 0x7f3f4c157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.701+0000 7f3f51e83640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c1610c0 0x7f3f4c1584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.701+0000 7f3f51e83640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c0d30f0 con 0x7f3f4c1610c0 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.701+0000 7f3f51e83640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c0d2e10 con 0x7f3f4c160800 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.701+0000 7f3f51e83640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4c0b6300 con 0x7f3f4c0d2320 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f4bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c1610c0 0x7f3f4c1584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f4bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c1610c0 0x7f3f4c1584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35922/0 (socket says 172.21.15.37:35922) 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f4bfff640 1 -- 172.21.15.37:0/2414292216 learned_addr learned my addr 172.21.15.37:0/2414292216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f4affd640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f4c160800 0x7f3f4c157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f4b7fe640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f4c0d2320 0x7f3f4c1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f4bfff640 1 -- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f4c0d2320 msgr2=0x7f3f4c1573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f4bfff640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f4c0d2320 0x7f3f4c1573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f4bfff640 1 -- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f4c160800 msgr2=0x7f3f4c157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f4bfff640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f4c160800 0x7f3f4c157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f4bfff640 1 -- 172.21.15.37:0/2414292216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c1e9e80 con 0x7f3f4c1610c0 2024-04-26T20:22:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f4bfff640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c1610c0 0x7f3f4c1584f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009eb0 tx=0x7f3f44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f33fff640 1 -- 172.21.15.37:0/2414292216 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44040070 con 0x7f3f4c1610c0 2024-04-26T20:22:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f33fff640 1 -- 172.21.15.37:0/2414292216 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f44031bc0 con 0x7f3f4c1610c0 2024-04-26T20:22:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f51e83640 1 -- 172.21.15.37:0/2414292216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4c1e9ba0 con 0x7f3f4c1610c0 2024-04-26T20:22:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f33fff640 1 -- 172.21.15.37:0/2414292216 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4403c420 con 0x7f3f4c1610c0 2024-04-26T20:22:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.702+0000 7f3f51e83640 1 -- 172.21.15.37:0/2414292216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4c144900 con 0x7f3f4c1610c0 2024-04-26T20:22:51.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.703+0000 7f3f51e83640 1 -- 172.21.15.37:0/2414292216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f4c06a360 con 0x7f3f4c1610c0 2024-04-26T20:22:51.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.704+0000 7f3f33fff640 1 -- 172.21.15.37:0/2414292216 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f44039040 con 0x7f3f4c1610c0 2024-04-26T20:22:51.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.704+0000 7f3f33fff640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3f3806c8f0 0x7f3f3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:51.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.704+0000 7f3f33fff640 1 -- 172.21.15.37:0/2414292216 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3f440ad040 con 0x7f3f4c1610c0 2024-04-26T20:22:51.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.704+0000 7f3f4b7fe640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3f3806c8f0 0x7f3f3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.705+0000 7f3f4b7fe640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3f3806c8f0 0x7f3f3806edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c1f8d50 tx=0x7f3f3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.708+0000 7f3f33fff640 1 -- 172.21.15.37:0/2414292216 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f4407b5c0 con 0x7f3f4c1610c0 2024-04-26T20:22:51.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.968+0000 7f3f51e83640 1 -- 172.21.15.37:0/2414292216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3f4c0d2ba0 con 0x7f3f4c1610c0 2024-04-26T20:22:51.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.970+0000 7f3f33fff640 1 -- 172.21.15.37:0/2414292216 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f3f4407aec0 con 0x7f3f4c1610c0 2024-04-26T20:22:51.971 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-26T20:22:51.971 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCkDCxmQTcnCBAAFh1Ew2ihHFtREYikzwF+Dw== 2024-04-26T20:22:51.971 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-04-26T20:22:51.971 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-04-26T20:22:51.971 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T20:22:51.972 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-04-26T20:22:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.972+0000 7f3f51e83640 1 -- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3f3806c8f0 msgr2=0x7f3f3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.972+0000 7f3f51e83640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3f3806c8f0 0x7f3f3806edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c1f8d50 tx=0x7f3f3c009040 comp rx=0 tx=0).stop 2024-04-26T20:22:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.972+0000 7f3f51e83640 1 -- 172.21.15.37:0/2414292216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c1610c0 msgr2=0x7f3f4c1584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.972+0000 7f3f51e83640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c1610c0 0x7f3f4c1584f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009eb0 tx=0x7f3f44004a40 comp rx=0 tx=0).stop 2024-04-26T20:22:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.972+0000 7f3f51e83640 1 -- 172.21.15.37:0/2414292216 shutdown_connections 2024-04-26T20:22:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.972+0000 7f3f51e83640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c1610c0 0x7f3f4c1584f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.972+0000 7f3f51e83640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f4c160800 0x7f3f4c157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.972+0000 7f3f51e83640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3f3806c8f0 0x7f3f3806edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.972+0000 7f3f51e83640 1 --2- 172.21.15.37:0/2414292216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f4c0d2320 0x7f3f4c1573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.972+0000 7f3f51e83640 1 -- 172.21.15.37:0/2414292216 >> 172.21.15.37:0/2414292216 conn(0x7f3f4c09e550 msgr2=0x7f3f4c161dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.972+0000 7f3f51e83640 1 -- 172.21.15.37:0/2414292216 shutdown_connections 2024-04-26T20:22:51.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:51.973+0000 7f3f51e83640 1 -- 172.21.15.37:0/2414292216 wait complete. 2024-04-26T20:22:51.984 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:22:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.084+0000 7f6b01eea640 1 Processor -- start 2024-04-26T20:22:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.084+0000 7f6b01eea640 1 -- start start 2024-04-26T20:22:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.085+0000 7f6b01eea640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6afc06ea40 0x7f6afc0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.085+0000 7f6b01eea640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6afc0b1d70 con 0x7f6afc0cf830 2024-04-26T20:22:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.085+0000 7f6b01eea640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6afc0d1960 con 0x7f6afc162e40 2024-04-26T20:22:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.085+0000 7f6b01eea640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6afc163220 con 0x7f6afc06ea40 2024-04-26T20:22:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.085+0000 7f6afbfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6afc0cf830 0x7f6afc0cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52618/0 (socket says 172.21.15.37:52618) 2024-04-26T20:22:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.085+0000 7f6afaffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6afc06ea40 0x7f6afc0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.085+0000 7f6afbfff640 1 -- 172.21.15.37:0/3395225935 learned_addr learned my addr 172.21.15.37:0/3395225935 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.086+0000 7f6afaffd640 1 -- 172.21.15.37:0/3395225935 >> v1:172.21.15.134:6789/0 conn(0x7f6afc162e40 legacy=0x7f6afc0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.086+0000 7f6afaffd640 1 -- 172.21.15.37:0/3395225935 >> v1:172.21.15.37:6789/0 conn(0x7f6afc0cf830 legacy=0x7f6afc0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.086+0000 7f6afaffd640 1 -- 172.21.15.37:0/3395225935 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6afc0d1c40 con 0x7f6afc06ea40 2024-04-26T20:22:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.086+0000 7f6afa7fc640 1 -- 172.21.15.37:0/3395225935 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3287828085 0 0) 0x7f6afc0b1d70 con 0x7f6afc0cf830 2024-04-26T20:22:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.086+0000 7f6afaffd640 1 --2- 172.21.15.37:0/3395225935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6afc06ea40 0x7f6afc0d1340 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6af00097a0 tx=0x7f6af002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba2336d1d355b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.086+0000 7f6afa7fc640 1 -- 172.21.15.37:0/3395225935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0037070 con 0x7f6afc06ea40 2024-04-26T20:22:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.087+0000 7f6afa7fc640 1 -- 172.21.15.37:0/3395225935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af0032440 con 0x7f6afc06ea40 2024-04-26T20:22:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.087+0000 7f6afa7fc640 1 -- 172.21.15.37:0/3395225935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0032810 con 0x7f6afc06ea40 2024-04-26T20:22:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.087+0000 7f6b01eea640 1 -- 172.21.15.37:0/3395225935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6afc06ea40 msgr2=0x7f6afc0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.087+0000 7f6b01eea640 1 --2- 172.21.15.37:0/3395225935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6afc06ea40 0x7f6afc0d1340 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6af00097a0 tx=0x7f6af002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.087+0000 7f6b01eea640 1 -- 172.21.15.37:0/3395225935 shutdown_connections 2024-04-26T20:22:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.087+0000 7f6b01eea640 1 --2- 172.21.15.37:0/3395225935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6afc06ea40 0x7f6afc0d1340 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.087+0000 7f6b01eea640 1 -- 172.21.15.37:0/3395225935 >> 172.21.15.37:0/3395225935 conn(0x7f6afc09ba50 msgr2=0x7f6afc09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.087+0000 7f6b01eea640 1 -- 172.21.15.37:0/3395225935 shutdown_connections 2024-04-26T20:22:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.087+0000 7f6b01eea640 1 -- 172.21.15.37:0/3395225935 wait complete. 2024-04-26T20:22:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.088+0000 7f6b01eea640 1 Processor -- start 2024-04-26T20:22:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.088+0000 7f6b01eea640 1 -- start start 2024-04-26T20:22:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.089+0000 7f6b01eea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc06ea40 0x7f6afc15f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.089+0000 7f6b01eea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6afc0cf830 0x7f6afc15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.089+0000 7f6b01eea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6afc162e40 0x7f6afc151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.089+0000 7f6b01eea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc0ce010 con 0x7f6afc162e40 2024-04-26T20:22:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.089+0000 7f6b01eea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc0d0eb0 con 0x7f6afc06ea40 2024-04-26T20:22:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.089+0000 7f6b01eea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6afc0ce190 con 0x7f6afc0cf830 2024-04-26T20:22:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.089+0000 7f6afbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6afc162e40 0x7f6afc151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.089+0000 7f6afbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6afc162e40 0x7f6afc151300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35940/0 (socket says 172.21.15.37:35940) 2024-04-26T20:22:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.089+0000 7f6afbfff640 1 -- 172.21.15.37:0/2735447438 learned_addr learned my addr 172.21.15.37:0/2735447438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6afaffd640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6afc0cf830 0x7f6afc15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6afb7fe640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc06ea40 0x7f6afc15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6afbfff640 1 -- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6afc0cf830 msgr2=0x7f6afc15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6afbfff640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6afc0cf830 0x7f6afc15dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6afbfff640 1 -- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc06ea40 msgr2=0x7f6afc15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6afbfff640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc06ea40 0x7f6afc15f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6afbfff640 1 -- 172.21.15.37:0/2735447438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6afc15e400 con 0x7f6afc162e40 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6afaffd640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6afc0cf830 0x7f6afc15dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6afb7fe640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc06ea40 0x7f6afc15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6afbfff640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6afc162e40 0x7f6afc151300 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6af400c7d0 tx=0x7f6af400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6ae3fff640 1 -- 172.21.15.37:0/2735447438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4018070 con 0x7f6afc162e40 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6ae3fff640 1 -- 172.21.15.37:0/2735447438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af400a5e0 con 0x7f6afc162e40 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6b01eea640 1 -- 172.21.15.37:0/2735447438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6afc15e680 con 0x7f6afc162e40 2024-04-26T20:22:52.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6ae3fff640 1 -- 172.21.15.37:0/2735447438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af400aa10 con 0x7f6afc162e40 2024-04-26T20:22:52.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.090+0000 7f6b01eea640 1 -- 172.21.15.37:0/2735447438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6afc15ec40 con 0x7f6afc162e40 2024-04-26T20:22:52.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.091+0000 7f6b01eea640 1 -- 172.21.15.37:0/2735447438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6afc06a360 con 0x7f6afc162e40 2024-04-26T20:22:52.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.092+0000 7f6ae3fff640 1 -- 172.21.15.37:0/2735447438 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6af401b070 con 0x7f6afc162e40 2024-04-26T20:22:52.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.092+0000 7f6ae3fff640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6ae806c8f0 0x7f6ae806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.092+0000 7f6ae3fff640 1 -- 172.21.15.37:0/2735447438 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6af408b090 con 0x7f6afc162e40 2024-04-26T20:22:52.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.092+0000 7f6afb7fe640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6ae806c8f0 0x7f6ae806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.093+0000 7f6afb7fe640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6ae806c8f0 0x7f6ae806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6aec000f60 tx=0x7f6aec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.096+0000 7f6ae3fff640 1 -- 172.21.15.37:0/2735447438 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6afc15e400 con 0x7f6afc162e40 2024-04-26T20:22:52.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.319+0000 7f6b01eea640 1 -- 172.21.15.37:0/2735447438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6afc0b4cd0 con 0x7f6afc162e40 2024-04-26T20:22:52.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.319+0000 7f6ae3fff640 1 -- 172.21.15.37:0/2735447438 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5181 (secure 0 0 0) 0x7f6af4059550 con 0x7f6afc162e40 2024-04-26T20:22:52.320 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:22:52.320 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/4166977617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4166977617},{"type":"v1","addr":"172.21.15.134:6836","nonce":4166977617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3245561282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3245561282},{"type":"v1","addr":"172.21.15.134:6837","nonce":3245561282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3853963851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3853963851},{"type":"v1","addr":"172.21.15.37:6835","nonce":3853963851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:22:42.116910+0000","modified":"2024-04-26T20:22:45.119356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5097":{"gid":5097,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/64661056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":64661056},{"type":"v1","addr":"172.21.15.37:6834","nonce":64661056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5097,"qdb_cluster":[5097]},"id":2}]} 2024-04-26T20:22:52.321 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 15 2024-04-26T20:22:52.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 -- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6ae806c8f0 msgr2=0x7f6ae806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6ae806c8f0 0x7f6ae806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6aec000f60 tx=0x7f6aec009040 comp rx=0 tx=0).stop 2024-04-26T20:22:52.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 -- 172.21.15.37:0/2735447438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6afc162e40 msgr2=0x7f6afc151300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6afc162e40 0x7f6afc151300 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6af400c7d0 tx=0x7f6af400cc90 comp rx=0 tx=0).stop 2024-04-26T20:22:52.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 -- 172.21.15.37:0/2735447438 shutdown_connections 2024-04-26T20:22:52.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6afc162e40 0x7f6afc151300 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6ae806c8f0 0x7f6ae806edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6afc0cf830 0x7f6afc15dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 --2- 172.21.15.37:0/2735447438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc06ea40 0x7f6afc15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 -- 172.21.15.37:0/2735447438 >> 172.21.15.37:0/2735447438 conn(0x7f6afc09ba50 msgr2=0x7f6afc09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:52.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 -- 172.21.15.37:0/2735447438 shutdown_connections 2024-04-26T20:22:52.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.321+0000 7f6b01eea640 1 -- 172.21.15.37:0/2735447438 wait complete. 2024-04-26T20:22:52.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:22:42.116910+0000', 'modified': '2024-04-26T20:22:45.119356+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5097}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5097': {'gid': 5097, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6834/64661056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 64661056}, {'type': 'v1', 'addr': '172.21.15.37:6834', 'nonce': 64661056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5097, 'qdb_cluster': [5097]} 2024-04-26T20:22:52.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:22:52.736 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T20:22:52.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.839+0000 7f65c361d640 1 Processor -- start 2024-04-26T20:22:52.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.840+0000 7f65c361d640 1 -- start start 2024-04-26T20:22:52.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.840+0000 7f65c361d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65bc162e40 0x7f65bc0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.840+0000 7f65c361d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f65bc0b2050 0x7f65bc0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.840+0000 7f65c361d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f65bc0cfd70 0x7f65bc0ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c361d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f65bc163220 con 0x7f65bc0cfd70 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c361d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f65bc0d0eb0 con 0x7f65bc0b2050 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c361d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f65bc0d01a0 con 0x7f65bc162e40 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c1bba640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f65bc0cfd70 0x7f65bc0ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c1bba640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f65bc0cfd70 0x7f65bc0ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35950/0 (socket says 172.21.15.37:35950) 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c1bba640 1 -- 172.21.15.37:0/178897027 learned_addr learned my addr 172.21.15.37:0/178897027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c13b9640 1 --2- 172.21.15.37:0/178897027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65bc162e40 0x7f65bc0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c0bb8640 1 --2- 172.21.15.37:0/178897027 >> v2:172.21.15.134:3300/0 conn(0x7f65bc0b2050 0x7f65bc0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c1bba640 1 -- 172.21.15.37:0/178897027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65bc162e40 msgr2=0x7f65bc0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c1bba640 1 --2- 172.21.15.37:0/178897027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65bc162e40 0x7f65bc0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c1bba640 1 -- 172.21.15.37:0/178897027 >> v2:172.21.15.134:3300/0 conn(0x7f65bc0b2050 msgr2=0x7f65bc0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c1bba640 1 --2- 172.21.15.37:0/178897027 >> v2:172.21.15.134:3300/0 conn(0x7f65bc0b2050 0x7f65bc0cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c1bba640 1 -- 172.21.15.37:0/178897027 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc0b1d70 con 0x7f65bc0cfd70 2024-04-26T20:22:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.841+0000 7f65c0bb8640 1 --2- 172.21.15.37:0/178897027 >> v2:172.21.15.134:3300/0 conn(0x7f65bc0b2050 0x7f65bc0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:52.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.842+0000 7f65c1bba640 1 --2- 172.21.15.37:0/178897027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc0cfd70 0x7f65bc0ce010 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65bc073b00 tx=0x7f65b402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f0ab439f38ca02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.842+0000 7f65b3fff640 1 -- 172.21.15.37:0/178897027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4036070 con 0x7f65bc0cfd70 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.842+0000 7f65b3fff640 1 -- 172.21.15.37:0/178897027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b4031600 con 0x7f65bc0cfd70 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.842+0000 7f65b3fff640 1 -- 172.21.15.37:0/178897027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40319d0 con 0x7f65bc0cfd70 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.842+0000 7f65c361d640 1 -- 172.21.15.37:0/178897027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc0cfd70 msgr2=0x7f65bc0ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.842+0000 7f65c361d640 1 --2- 172.21.15.37:0/178897027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc0cfd70 0x7f65bc0ce010 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65bc073b00 tx=0x7f65b402a610 comp rx=0 tx=0).stop 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.842+0000 7f65c361d640 1 -- 172.21.15.37:0/178897027 shutdown_connections 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.842+0000 7f65c361d640 1 --2- 172.21.15.37:0/178897027 >> v2:172.21.15.134:3300/0 conn(0x7f65bc0b2050 0x7f65bc0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.842+0000 7f65c361d640 1 --2- 172.21.15.37:0/178897027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc0cfd70 0x7f65bc0ce010 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.842+0000 7f65c361d640 1 --2- 172.21.15.37:0/178897027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65bc162e40 0x7f65bc0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.842+0000 7f65c361d640 1 -- 172.21.15.37:0/178897027 >> 172.21.15.37:0/178897027 conn(0x7f65bc09ba50 msgr2=0x7f65bc09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.843+0000 7f65c361d640 1 -- 172.21.15.37:0/178897027 shutdown_connections 2024-04-26T20:22:52.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.843+0000 7f65c361d640 1 -- 172.21.15.37:0/178897027 wait complete. 2024-04-26T20:22:52.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.844+0000 7f65c361d640 1 Processor -- start 2024-04-26T20:22:52.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.844+0000 7f65c361d640 1 -- start start 2024-04-26T20:22:52.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.844+0000 7f65c361d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65bc0b2050 0x7f65bc0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.844+0000 7f65c361d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65bc0cfd70 0x7f65bc0c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.844+0000 7f65c361d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc162e40 0x7f65bc15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.844+0000 7f65c361d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc0d1f20 con 0x7f65bc162e40 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.844+0000 7f65c361d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc0d1c40 con 0x7f65bc0cfd70 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.844+0000 7f65c361d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f65bc0b24b0 con 0x7f65bc0b2050 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c1bba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc162e40 0x7f65bc15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c1bba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc162e40 0x7f65bc15c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35960/0 (socket says 172.21.15.37:35960) 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c1bba640 1 -- 172.21.15.37:0/2084053944 learned_addr learned my addr 172.21.15.37:0/2084053944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c1bba640 1 -- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65bc0b2050 msgr2=0x7f65bc0cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c13b9640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65bc0b2050 0x7f65bc0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c0bb8640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65bc0cfd70 0x7f65bc0c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c1bba640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65bc0b2050 0x7f65bc0cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c1bba640 1 -- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65bc0cfd70 msgr2=0x7f65bc0c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c1bba640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65bc0cfd70 0x7f65bc0c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c1bba640 1 -- 172.21.15.37:0/2084053944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc0c9470 con 0x7f65bc162e40 2024-04-26T20:22:52.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c13b9640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65bc0b2050 0x7f65bc0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:52.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c0bb8640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65bc0cfd70 0x7f65bc0c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:52.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c1bba640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc162e40 0x7f65bc15c090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f65b4034040 tx=0x7f65b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65b1ffb640 1 -- 172.21.15.37:0/2084053944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4036050 con 0x7f65bc162e40 2024-04-26T20:22:52.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65b1ffb640 1 -- 172.21.15.37:0/2084053944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b4042070 con 0x7f65bc162e40 2024-04-26T20:22:52.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65c361d640 1 -- 172.21.15.37:0/2084053944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc0c8ec0 con 0x7f65bc162e40 2024-04-26T20:22:52.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.845+0000 7f65b1ffb640 1 -- 172.21.15.37:0/2084053944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4009510 con 0x7f65bc162e40 2024-04-26T20:22:52.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.846+0000 7f65c361d640 1 -- 172.21.15.37:0/2084053944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc160b60 con 0x7f65bc162e40 2024-04-26T20:22:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.847+0000 7f65b1ffb640 1 -- 172.21.15.37:0/2084053944 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65b4007bd0 con 0x7f65bc162e40 2024-04-26T20:22:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.847+0000 7f65b1ffb640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f65a006c8f0 0x7f65a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.847+0000 7f65b1ffb640 1 -- 172.21.15.37:0/2084053944 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f65b40aca20 con 0x7f65bc162e40 2024-04-26T20:22:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.848+0000 7f65c13b9640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f65a006c8f0 0x7f65a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.848+0000 7f65c13b9640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f65a006c8f0 0x7f65a006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006fd0 tx=0x7f65b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.848+0000 7f65c6410740 1 Processor -- start 2024-04-26T20:22:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.848+0000 7f65c6410740 1 -- start start 2024-04-26T20:22:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.849+0000 7f65c6410740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x558282548820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.849+0000 7f65c6410740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558282565ce0 0x558282566230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.849+0000 7f65c6410740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828251e120 0x5582825643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.849+0000 7f65c6410740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558282482740 con 0x55828251e120 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.849+0000 7f65c6410740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55828241e3a0 con 0x558282548440 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.849+0000 7f65c6410740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55828247e700 con 0x558282565ce0 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.849+0000 7f65c1bba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828251e120 0x5582825643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.849+0000 7f65c1bba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828251e120 0x5582825643a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35964/0 (socket says 172.21.15.37:35964) 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.849+0000 7f65c1bba640 1 -- 172.21.15.37:0/1256554380 learned_addr learned my addr 172.21.15.37:0/1256554380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.849+0000 7f65c13b9640 1 --2- 172.21.15.37:0/1256554380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x558282548820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.849+0000 7f65c0bb8640 1 --2- 172.21.15.37:0/1256554380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558282565ce0 0x558282566230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c1bba640 1 -- 172.21.15.37:0/1256554380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558282565ce0 msgr2=0x558282566230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c1bba640 1 --2- 172.21.15.37:0/1256554380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558282565ce0 0x558282566230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c1bba640 1 -- 172.21.15.37:0/1256554380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 msgr2=0x558282548820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c1bba640 1 --2- 172.21.15.37:0/1256554380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x558282548820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c1bba640 1 -- 172.21.15.37:0/1256554380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55828243a750 con 0x55828251e120 2024-04-26T20:22:52.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c1bba640 1 --2- 172.21.15.37:0/1256554380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828251e120 0x5582825643a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f65b407a6d0 tx=0x7f65b407a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60dc45f2b7b70a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65a6ffd640 1 -- 172.21.15.37:0/1256554380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40a9070 con 0x55828251e120 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65a6ffd640 1 -- 172.21.15.37:0/1256554380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b4037040 con 0x55828251e120 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65a6ffd640 1 -- 172.21.15.37:0/1256554380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b407ae40 con 0x55828251e120 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c6410740 1 -- 172.21.15.37:0/1256554380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828251e120 msgr2=0x5582825643a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c6410740 1 --2- 172.21.15.37:0/1256554380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828251e120 0x5582825643a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f65b407a6d0 tx=0x7f65b407a0a0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c6410740 1 -- 172.21.15.37:0/1256554380 shutdown_connections 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c6410740 1 --2- 172.21.15.37:0/1256554380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828251e120 0x5582825643a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c6410740 1 --2- 172.21.15.37:0/1256554380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558282565ce0 0x558282566230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c6410740 1 --2- 172.21.15.37:0/1256554380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x558282548820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.850+0000 7f65c6410740 1 -- 172.21.15.37:0/1256554380 >> 172.21.15.37:0/1256554380 conn(0x55828251aa50 msgr2=0x558282563250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.851+0000 7f65c6410740 1 -- 172.21.15.37:0/1256554380 shutdown_connections 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.851+0000 7f65c6410740 1 -- 172.21.15.37:0/1256554380 wait complete. 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.851+0000 7f65c6410740 1 -- start start 2024-04-26T20:22:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.851+0000 7f65c6410740 1 Processor -- start 2024-04-26T20:22:52.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c6410740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x558282517670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c6410740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282517bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c6410740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55828255a300 0x55828254a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c6410740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558282482740 con 0x558282565ce0 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c6410740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558282409fe0 con 0x558282548440 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c6410740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55828251eec0 con 0x55828255a300 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c6410740 10 client.0 _ll_register_callbacks cb 0x558282420150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c0bb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282517bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c0bb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282517bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35970/0 (socket says 172.21.15.37:35970) 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c0bb8640 1 -- 172.21.15.37:0/1061195529 learned_addr learned my addr 172.21.15.37:0/1061195529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c13b9640 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x558282517670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c1bba640 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55828255a300 0x55828254a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c0bb8640 1 -- 172.21.15.37:0/1061195529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55828255a300 msgr2=0x55828254a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.852+0000 7f65c0bb8640 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55828255a300 0x55828254a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c0bb8640 1 -- 172.21.15.37:0/1061195529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 msgr2=0x558282517670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c0bb8640 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x558282517670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c0bb8640 1 -- 172.21.15.37:0/1061195529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582821942d0 con 0x558282565ce0 2024-04-26T20:22:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c0bb8640 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282517bb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f65ac010590 tx=0x7f65ac010a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9774339a41b52e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f659bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f659bfff640 1 -- 172.21.15.37:0/1061195529 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac012860 con 0x558282565ce0 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f659bfff640 1 -- 172.21.15.37:0/1061195529 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f65ac012a00 con 0x558282565ce0 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c6410740 1 -- 172.21.15.37:0/1061195529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b4037040 con 0x558282565ce0 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f659bfff640 1 -- 172.21.15.37:0/1061195529 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f65ac012c00 con 0x558282565ce0 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c6410740 10 client.5310 fetch_fsmap learned FSMap version 15 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c6410740 1 -- 172.21.15.37:0/1061195529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55828225e6b0 con 0x558282565ce0 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f659bfff640 1 -- 172.21.15.37:0/1061195529 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b4037040 con 0x558282565ce0 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f659bfff640 1 -- 172.21.15.37:0/1061195529 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f65ac005990 con 0x558282565ce0 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c6410740 10 client.5310 fetch_fsmap finished waiting for FSMap version 15 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c6410740 10 client.5310 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c6410740 20 client.5310 populate_metadata read hostname 'smithi037' 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c6410740 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.37:6832/64661056,v1:172.21.15.37:6834/64661056] conn(0x558282573b50 0x558282573f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c6410740 4 client.5310 mds_command: new command op to 5097 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:22:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.853+0000 7f65c6410740 1 -- 172.21.15.37:0/1061195529 --> [v2:172.21.15.37:6832/64661056,v1:172.21.15.37:6834/64661056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558282490350 con 0x558282573b50 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.854+0000 7f65c13b9640 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.37:6832/64661056,v1:172.21.15.37:6834/64661056] conn(0x558282573b50 0x558282573f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.854+0000 7f659bfff640 10 client.5310 ms_handle_connect on v2:172.21.15.37:6832/64661056 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.854+0000 7f65c13b9640 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.37:6832/64661056,v1:172.21.15.37:6834/64661056] conn(0x558282573b50 0x558282573f30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca800dc6103d388 server_cookie=5077f771264db2cd in_seq=0 out_seq=0 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f659bfff640 1 -- 172.21.15.37:0/1061195529 <== mds.0 v2:172.21.15.37:6832/64661056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558282490350 con 0x558282573b50 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f659bfff640 10 client.5310 handle_command_reply: tid=0 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 1 client.5310 shutdown 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 1 -- 172.21.15.37:0/1061195529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 msgr2=0x558282517bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282517bb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f65ac010590 tx=0x7f65ac010a50 comp rx=0 tx=0).stop 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 1 -- 172.21.15.37:0/1061195529 shutdown_connections 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55828255a300 0x55828254a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.37:6832/64661056,v1:172.21.15.37:6834/64661056] conn(0x558282573b50 0x558282573f30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282517bb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 1 --2- 172.21.15.37:0/1061195529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x558282517670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f659bfff640 0 client.5310 ms_handle_reset on v2:172.21.15.37:6832/64661056 2024-04-26T20:22:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 1 -- 172.21.15.37:0/1061195529 >> 172.21.15.37:0/1061195529 conn(0x55828251e120 msgr2=0x5582825667d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 1 -- 172.21.15.37:0/1061195529 shutdown_connections 2024-04-26T20:22:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 1 -- 172.21.15.37:0/1061195529 wait complete. 2024-04-26T20:22:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.855+0000 7f65c6410740 20 client.5310 trim_cache size 0 max 16384 2024-04-26T20:22:52.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.873+0000 7f65c6410740 1 Processor -- start 2024-04-26T20:22:52.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.873+0000 7f65c6410740 1 -- start start 2024-04-26T20:22:52.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c6410740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x55828257ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c6410740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55828255a300 0x55828257cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c6410740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282558be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c6410740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc200920 con 0x558282565ce0 2024-04-26T20:22:52.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c6410740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc0b3210 con 0x558282548440 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c6410740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f65bc0b3860 con 0x55828255a300 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c1bba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282558be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c1bba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282558be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35978/0 (socket says 172.21.15.37:35978) 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c1bba640 1 -- 172.21.15.37:0/719745818 learned_addr learned my addr 172.21.15.37:0/719745818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c1bba640 1 -- 172.21.15.37:0/719745818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55828255a300 msgr2=0x55828257cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c13b9640 1 --2- 172.21.15.37:0/719745818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x55828257ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c1bba640 1 --2- 172.21.15.37:0/719745818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55828255a300 0x55828257cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c1bba640 1 -- 172.21.15.37:0/719745818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 msgr2=0x55828257ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c1bba640 1 --2- 172.21.15.37:0/719745818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x55828257ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c1bba640 1 -- 172.21.15.37:0/719745818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558282513ea0 con 0x558282565ce0 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.874+0000 7f65c1bba640 1 --2- 172.21.15.37:0/719745818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282558be0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f65b407a6d0 tx=0x7f65b407c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7dd1857297ab15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65a67fc640 1 -- 172.21.15.37:0/719745818 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40a9040 con 0x558282565ce0 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65a67fc640 1 -- 172.21.15.37:0/719745818 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b407ae40 con 0x558282565ce0 2024-04-26T20:22:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65a67fc640 1 -- 172.21.15.37:0/719745818 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40769e0 con 0x558282565ce0 2024-04-26T20:22:52.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65c6410740 1 -- 172.21.15.37:0/719745818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 msgr2=0x558282558be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65c6410740 1 --2- 172.21.15.37:0/719745818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282558be0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f65b407a6d0 tx=0x7f65b407c000 comp rx=0 tx=0).stop 2024-04-26T20:22:52.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65c6410740 1 -- 172.21.15.37:0/719745818 shutdown_connections 2024-04-26T20:22:52.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65c6410740 1 --2- 172.21.15.37:0/719745818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558282565ce0 0x558282558be0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65c6410740 1 --2- 172.21.15.37:0/719745818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55828255a300 0x55828257cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65c6410740 1 --2- 172.21.15.37:0/719745818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x55828257ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65c6410740 1 -- 172.21.15.37:0/719745818 >> 172.21.15.37:0/719745818 conn(0x55828251e120 msgr2=0x558282517890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:52.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65c6410740 1 -- 172.21.15.37:0/719745818 shutdown_connections 2024-04-26T20:22:52.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.875+0000 7f65c6410740 1 -- 172.21.15.37:0/719745818 wait complete. 2024-04-26T20:22:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.876+0000 7f65c6410740 1 -- start start 2024-04-26T20:22:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.876+0000 7f65c6410740 1 Processor -- start 2024-04-26T20:22:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c6410740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x5582825d7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c6410740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828255a300 0x5582825713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c6410740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558282565ce0 0x558282571920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c6410740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558282544c70 con 0x55828255a300 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c6410740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558282573060 con 0x558282548440 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c6410740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55828254ae50 con 0x558282565ce0 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c6410740 10 client.0 _ll_register_callbacks cb 0x7f65ac00b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c0bb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828255a300 0x5582825713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c0bb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828255a300 0x5582825713e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35992/0 (socket says 172.21.15.37:35992) 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c0bb8640 1 -- 172.21.15.37:0/4097334050 learned_addr learned my addr 172.21.15.37:0/4097334050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c1bba640 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558282565ce0 0x558282571920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c13b9640 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x5582825d7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c0bb8640 1 -- 172.21.15.37:0/4097334050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558282565ce0 msgr2=0x558282571920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c0bb8640 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558282565ce0 0x558282571920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c0bb8640 1 -- 172.21.15.37:0/4097334050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 msgr2=0x5582825d7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c0bb8640 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x5582825d7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c0bb8640 1 -- 172.21.15.37:0/4097334050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558282566cf0 con 0x55828255a300 2024-04-26T20:22:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f65c0bb8640 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828255a300 0x5582825713e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f65ac010a90 tx=0x7f65ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303715bb200badcf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f659bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.877+0000 7f659bfff640 1 -- 172.21.15.37:0/4097334050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac02c020 con 0x55828255a300 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f659bfff640 1 -- 172.21.15.37:0/4097334050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f65ac028450 con 0x55828255a300 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f65c6410740 1 -- 172.21.15.37:0/4097334050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b407ae40 con 0x55828255a300 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f659bfff640 1 -- 172.21.15.37:0/4097334050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f65ac036050 con 0x55828255a300 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f65c6410740 10 client.5316 fetch_fsmap learned FSMap version 15 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f65c6410740 1 -- 172.21.15.37:0/4097334050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55828251c2b0 con 0x55828255a300 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f659bfff640 1 -- 172.21.15.37:0/4097334050 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b407ae40 con 0x55828255a300 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f659bfff640 1 -- 172.21.15.37:0/4097334050 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f65ac039020 con 0x55828255a300 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f65c6410740 10 client.5316 fetch_fsmap finished waiting for FSMap version 15 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f65c6410740 10 client.5316 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f65c6410740 20 client.5316 populate_metadata read hostname 'smithi037' 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f65c6410740 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.37:6832/64661056,v1:172.21.15.37:6834/64661056] conn(0x5582825e1230 0x5582825e1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f65c6410740 4 client.5316 mds_command: new command op to 5097 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:22:52.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f65c6410740 1 -- 172.21.15.37:0/4097334050 --> [v2:172.21.15.37:6832/64661056,v1:172.21.15.37:6834/64661056] -- command(tid 0: {"prefix": "status"}) -- 0x558282519870 con 0x5582825e1230 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f65c13b9640 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.37:6832/64661056,v1:172.21.15.37:6834/64661056] conn(0x5582825e1230 0x5582825e1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f65c13b9640 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.37:6832/64661056,v1:172.21.15.37:6834/64661056] conn(0x5582825e1230 0x5582825e1610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b691d1e364a6aae server_cookie=ff4763eb52c7b95 in_seq=0 out_seq=0 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.878+0000 7f659bfff640 10 client.5316 ms_handle_connect on v2:172.21.15.37:6832/64661056 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f659bfff640 1 -- 172.21.15.37:0/4097334050 <== mds.0 v2:172.21.15.37:6832/64661056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x558282519870 con 0x5582825e1230 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f659bfff640 10 client.5316 handle_command_reply: tid=0 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 1 client.5316 shutdown 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 1 -- 172.21.15.37:0/4097334050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828255a300 msgr2=0x5582825713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828255a300 0x5582825713e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f65ac010a90 tx=0x7f65ac004730 comp rx=0 tx=0).stop 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 1 -- 172.21.15.37:0/4097334050 shutdown_connections 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558282565ce0 0x558282571920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.37:6832/64661056,v1:172.21.15.37:6834/64661056] conn(0x5582825e1230 0x5582825e1610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55828255a300 0x5582825713e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 1 --2- 172.21.15.37:0/4097334050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558282548440 0x5582825d7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 1 -- 172.21.15.37:0/4097334050 >> 172.21.15.37:0/4097334050 conn(0x55828251e120 msgr2=0x55828254a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f659bfff640 0 client.5316 ms_handle_reset on v2:172.21.15.37:6832/64661056 2024-04-26T20:22:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 1 -- 172.21.15.37:0/4097334050 shutdown_connections 2024-04-26T20:22:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 1 -- 172.21.15.37:0/4097334050 wait complete. 2024-04-26T20:22:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.879+0000 7f65c6410740 20 client.5316 trim_cache size 0 max 16384 2024-04-26T20:22:52.886 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:22:52.886 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "d49b1c18-e1bf-49f2-acda-0904fa7c6684", 2024-04-26T20:22:52.886 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-26T20:22:52.886 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5097, 2024-04-26T20:22:52.886 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T20:22:52.886 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T20:22:52.886 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T20:22:52.887 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 10.140444974999999, 2024-04-26T20:22:52.887 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 15, 2024-04-26T20:22:52.887 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 33, 2024-04-26T20:22:52.887 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 33, 2024-04-26T20:22:52.887 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 18.027144252999999 2024-04-26T20:22:52.887 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 -- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f65a006c8f0 msgr2=0x7f65a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f65a006c8f0 0x7f65a006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006fd0 tx=0x7f65b8008040 comp rx=0 tx=0).stop 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 -- 172.21.15.37:0/2084053944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc162e40 msgr2=0x7f65bc15c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc162e40 0x7f65bc15c090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f65b4034040 tx=0x7f65b4004a40 comp rx=0 tx=0).stop 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 -- 172.21.15.37:0/2084053944 shutdown_connections 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc162e40 0x7f65bc15c090 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65bc0cfd70 0x7f65bc0c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f65a006c8f0 0x7f65a006edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 --2- 172.21.15.37:0/2084053944 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65bc0b2050 0x7f65bc0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 -- 172.21.15.37:0/2084053944 >> 172.21.15.37:0/2084053944 conn(0x7f65bc09ba50 msgr2=0x7f65bc070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 -- 172.21.15.37:0/2084053944 shutdown_connections 2024-04-26T20:22:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.880+0000 7f65c361d640 1 -- 172.21.15.37:0/2084053944 wait complete. 2024-04-26T20:22:52.891 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:22:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.990+0000 7fb55a237640 1 Processor -- start 2024-04-26T20:22:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.990+0000 7fb55a237640 1 -- start start 2024-04-26T20:22:52.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.991+0000 7fb55a237640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb5540d3c80 0x7fb5540b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.991+0000 7fb55a237640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5540d0210 con 0x7fb5540d3820 2024-04-26T20:22:52.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.991+0000 7fb55a237640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5540d04f0 con 0x7fb554162b40 2024-04-26T20:22:52.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.991+0000 7fb55a237640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb5540d07d0 con 0x7fb5540d3c80 2024-04-26T20:22:52.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.991+0000 7fb553fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb5540d3c80 0x7fb5540b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.991+0000 7fb552ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb5540d3820 0x7fb5540b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52624/0 (socket says 172.21.15.37:52624) 2024-04-26T20:22:52.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.991+0000 7fb552ffd640 1 -- 172.21.15.37:0/1048276272 learned_addr learned my addr 172.21.15.37:0/1048276272 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:52.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.992+0000 7fb553fff640 1 -- 172.21.15.37:0/1048276272 >> v1:172.21.15.134:6789/0 conn(0x7fb554162b40 legacy=0x7fb5540d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:52.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.992+0000 7fb553fff640 1 -- 172.21.15.37:0/1048276272 >> v1:172.21.15.37:6789/0 conn(0x7fb5540d3820 legacy=0x7fb5540b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.992+0000 7fb553fff640 1 -- 172.21.15.37:0/1048276272 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5540d0ab0 con 0x7fb5540d3c80 2024-04-26T20:22:52.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.992+0000 7fb553fff640 1 --2- 172.21.15.37:0/1048276272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5540d3c80 0x7fb5540b52b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009d90 tx=0x7fb54c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a26e3bcb5c9512d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.993+0000 7fb5527fc640 1 -- 172.21.15.37:0/1048276272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c007740 con 0x7fb5540d3c80 2024-04-26T20:22:52.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.993+0000 7fb5527fc640 1 -- 172.21.15.37:0/1048276272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb54c0078e0 con 0x7fb5540d3c80 2024-04-26T20:22:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.993+0000 7fb5527fc640 1 -- 172.21.15.37:0/1048276272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c007cb0 con 0x7fb5540d3c80 2024-04-26T20:22:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.993+0000 7fb55a237640 1 -- 172.21.15.37:0/1048276272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5540d3c80 msgr2=0x7fb5540b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.993+0000 7fb55a237640 1 --2- 172.21.15.37:0/1048276272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5540d3c80 0x7fb5540b52b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009d90 tx=0x7fb54c02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.993+0000 7fb55a237640 1 -- 172.21.15.37:0/1048276272 shutdown_connections 2024-04-26T20:22:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.993+0000 7fb55a237640 1 --2- 172.21.15.37:0/1048276272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5540d3c80 0x7fb5540b52b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.993+0000 7fb55a237640 1 -- 172.21.15.37:0/1048276272 >> 172.21.15.37:0/1048276272 conn(0x7fb55409e550 msgr2=0x7fb5540d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.993+0000 7fb55a237640 1 -- 172.21.15.37:0/1048276272 shutdown_connections 2024-04-26T20:22:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.993+0000 7fb55a237640 1 -- 172.21.15.37:0/1048276272 wait complete. 2024-04-26T20:22:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.994+0000 7fb55a237640 1 Processor -- start 2024-04-26T20:22:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.994+0000 7fb55a237640 1 -- start start 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.995+0000 7fb55a237640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5540d3820 0x7fb5540cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.995+0000 7fb55a237640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5540d3c80 0x7fb5540c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.995+0000 7fb55a237640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb554162b40 0x7fb5540ba820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.995+0000 7fb55a237640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5540d12b0 con 0x7fb5540d3c80 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.995+0000 7fb55a237640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5540d4110 con 0x7fb554162b40 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.995+0000 7fb55a237640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb5540cfe30 con 0x7fb5540d3820 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.995+0000 7fb552ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5540d3c80 0x7fb5540c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.995+0000 7fb552ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5540d3c80 0x7fb5540c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36008/0 (socket says 172.21.15.37:36008) 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.995+0000 7fb552ffd640 1 -- 172.21.15.37:0/1595059900 learned_addr learned my addr 172.21.15.37:0/1595059900 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.995+0000 7fb5537fe640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5540d3820 0x7fb5540cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb553fff640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb554162b40 0x7fb5540ba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb552ffd640 1 -- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5540d3820 msgr2=0x7fb5540cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb552ffd640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5540d3820 0x7fb5540cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb552ffd640 1 -- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb554162b40 msgr2=0x7fb5540ba820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb552ffd640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb554162b40 0x7fb5540ba820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb552ffd640 1 -- 172.21.15.37:0/1595059900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5540c9e60 con 0x7fb5540d3c80 2024-04-26T20:22:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb552ffd640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5540d3c80 0x7fb5540c9920 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c3c0 tx=0x7fb54800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb53bfff640 1 -- 172.21.15.37:0/1595059900 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548017070 con 0x7fb5540d3c80 2024-04-26T20:22:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb53bfff640 1 -- 172.21.15.37:0/1595059900 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb548005ce0 con 0x7fb5540d3c80 2024-04-26T20:22:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb55a237640 1 -- 172.21.15.37:0/1595059900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5540bad60 con 0x7fb5540d3c80 2024-04-26T20:22:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb53bfff640 1 -- 172.21.15.37:0/1595059900 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5480072f0 con 0x7fb5540d3c80 2024-04-26T20:22:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.996+0000 7fb55a237640 1 -- 172.21.15.37:0/1595059900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5540ca3e0 con 0x7fb5540d3c80 2024-04-26T20:22:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.997+0000 7fb55a237640 1 -- 172.21.15.37:0/1595059900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb55406a360 con 0x7fb5540d3c80 2024-04-26T20:22:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.997+0000 7fb53bfff640 1 -- 172.21.15.37:0/1595059900 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb54801a070 con 0x7fb5540d3c80 2024-04-26T20:22:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.998+0000 7fb53bfff640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb54006c8f0 0x7fb54006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.998+0000 7fb53bfff640 1 -- 172.21.15.37:0/1595059900 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb54808ae10 con 0x7fb5540d3c80 2024-04-26T20:22:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.998+0000 7fb5537fe640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb54006c8f0 0x7fb54006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:52.999+0000 7fb5537fe640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb54006c8f0 0x7fb54006edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb544004ea0 tx=0x7fb544009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.002+0000 7fb53bfff640 1 -- 172.21.15.37:0/1595059900 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5480592d0 con 0x7fb5540d3c80 2024-04-26T20:22:53.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.227+0000 7fb55a237640 1 -- 172.21.15.37:0/1595059900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb5540d2960 con 0x7fb5540d3c80 2024-04-26T20:22:53.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.227+0000 7fb53bfff640 1 -- 172.21.15.37:0/1595059900 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7fb548058bd0 con 0x7fb5540d3c80 2024-04-26T20:22:53.229 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:22:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.229+0000 7fb55a237640 1 -- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb54006c8f0 msgr2=0x7fb54006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.229+0000 7fb55a237640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb54006c8f0 0x7fb54006edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb544004ea0 tx=0x7fb544009040 comp rx=0 tx=0).stop 2024-04-26T20:22:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.229+0000 7fb55a237640 1 -- 172.21.15.37:0/1595059900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5540d3c80 msgr2=0x7fb5540c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.229+0000 7fb55a237640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5540d3c80 0x7fb5540c9920 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c3c0 tx=0x7fb54800c880 comp rx=0 tx=0).stop 2024-04-26T20:22:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.229+0000 7fb55a237640 1 -- 172.21.15.37:0/1595059900 shutdown_connections 2024-04-26T20:22:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.229+0000 7fb55a237640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb554162b40 0x7fb5540ba820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.229+0000 7fb55a237640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5540d3c80 0x7fb5540c9920 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.229+0000 7fb55a237640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb54006c8f0 0x7fb54006edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.229+0000 7fb55a237640 1 --2- 172.21.15.37:0/1595059900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5540d3820 0x7fb5540cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.229+0000 7fb55a237640 1 -- 172.21.15.37:0/1595059900 >> 172.21.15.37:0/1595059900 conn(0x7fb55409e550 msgr2=0x7fb55415e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.230+0000 7fb55a237640 1 -- 172.21.15.37:0/1595059900 shutdown_connections 2024-04-26T20:22:53.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53.230+0000 7fb55a237640 1 -- 172.21.15.37:0/1595059900 wait complete. 2024-04-26T20:22:53.241 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T20:22:53.241 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:22:53.242 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-26T20:22:53.242 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:22:53.242 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:22:53.242 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T20:22:53.242 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T20:22:53.242 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:22:53.242 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:22:53.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:53.242 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-26T20:22:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:22:53.258 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:22:53.258 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:22:53.258 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:53.258 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-26T20:22:53.258 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:53.258 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42671sec preferred_lft 42671sec 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-26T20:22:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 06:a8:3c:e3:8e:a4 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::247e:2bff:fe9c:6859/64 scope link 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 06:a8:3c:e3:8e:a4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::4a8:3cff:fee3:8ea4/64 scope link 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 96:92:77:52:91:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::9492:77ff:fe52:9118/64 scope link 2024-04-26T20:22:53.260 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:53.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:53.261 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:22:53.321 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-26T20:22:53.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:53.322 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-26T20:22:53.379 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-26T20:22:53.379 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:22:53.379 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:22:53.379 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:22:53.379 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:22:53.379 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:53.380 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T20:22:53.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:22:53.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:22:53.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:22:53.380 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T20:22:53.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:53.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:53.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:53.506 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T20:22:53.521 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:22:53.521 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:22:53.521 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:22:53.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:53.521 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-26T20:22:53.585 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:53.585 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:22:53.641 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:22:53.641 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:53.641 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:22:53.696 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T20:22:53.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:53.696 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:53.778 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:53.779 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:53.786 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:53.786 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:53.786 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-04-26T20:22:53.786 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with new device syntax: 0@d49b1c18-e1bf-49f2-acda-0904fa7c6684.cephfs=/ 2024-04-26T20:22:53.786 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.37:3300/172.21.15.134:3300/172.21.15.134:3301" will pass to kernel 2024-04-26T20:22:53.787 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T20:22:53.787 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:22:53.787 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:22:53.787 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:22:53.787 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:22:53.789 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:22:53.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:53.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:22:53.854 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:53.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:53.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:53.959 INFO:teuthology.orchestra.run.smithi037.stdout:client5328 2024-04-26T20:22:53.960 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:53.960 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:53.975 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:22:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.115+0000 7f2bf4ac8640 1 Processor -- start 2024-04-26T20:22:54.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.115+0000 7f2bf4ac8640 1 -- start start 2024-04-26T20:22:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.116+0000 7f2bf4ac8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2bf006c240 0x7f2bf006fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.116+0000 7f2bf4ac8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2bf015f4f0 0x7f2bf006c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.116+0000 7f2bf4ac8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00b1480 0x7f2bf00ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.116+0000 7f2bf4ac8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf006cb60 con 0x7f2bf015f4f0 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.116+0000 7f2bf4ac8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf00b59b0 con 0x7f2bf006c240 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.116+0000 7f2bf4ac8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf0070350 con 0x7f2bf00b1480 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.116+0000 7f2be7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2bf015f4f0 0x7f2bf006c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.116+0000 7f2be7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2bf015f4f0 0x7f2bf006c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36018/0 (socket says 172.21.15.37:36018) 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.116+0000 7f2be7fff640 1 -- 172.21.15.37:0/1675913000 learned_addr learned my addr 172.21.15.37:0/1675913000 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.116+0000 7f2beed9d640 1 --2- 172.21.15.37:0/1675913000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00b1480 0x7f2bf00ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.117+0000 7f2bee59c640 1 --2- 172.21.15.37:0/1675913000 >> v2:172.21.15.134:3300/0 conn(0x7f2bf006c240 0x7f2bf006fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.117+0000 7f2be7fff640 1 -- 172.21.15.37:0/1675913000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00b1480 msgr2=0x7f2bf00ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.117+0000 7f2be7fff640 1 --2- 172.21.15.37:0/1675913000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00b1480 0x7f2bf00ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.117+0000 7f2be7fff640 1 -- 172.21.15.37:0/1675913000 >> v2:172.21.15.134:3300/0 conn(0x7f2bf006c240 msgr2=0x7f2bf006fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.117+0000 7f2be7fff640 1 --2- 172.21.15.37:0/1675913000 >> v2:172.21.15.134:3300/0 conn(0x7f2bf006c240 0x7f2bf006fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.117+0000 7f2be7fff640 1 -- 172.21.15.37:0/1675913000 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf00b5c70 con 0x7f2bf015f4f0 2024-04-26T20:22:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.117+0000 7f2bee59c640 1 --2- 172.21.15.37:0/1675913000 >> v2:172.21.15.134:3300/0 conn(0x7f2bf006c240 0x7f2bf006fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.117+0000 7f2be7fff640 1 --2- 172.21.15.37:0/1675913000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf015f4f0 0x7f2bf006c620 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00b1ab0 tx=0x7f2bd802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ffc6eceabd8d55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.117+0000 7f2bedd9b640 1 -- 172.21.15.37:0/1675913000 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8037070 con 0x7f2bf015f4f0 2024-04-26T20:22:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.117+0000 7f2bedd9b640 1 -- 172.21.15.37:0/1675913000 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd8032440 con 0x7f2bf015f4f0 2024-04-26T20:22:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.117+0000 7f2bedd9b640 1 -- 172.21.15.37:0/1675913000 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8032810 con 0x7f2bf015f4f0 2024-04-26T20:22:54.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.118+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/1675913000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf015f4f0 msgr2=0x7f2bf006c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:54.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.118+0000 7f2bf4ac8640 1 --2- 172.21.15.37:0/1675913000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf015f4f0 0x7f2bf006c620 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00b1ab0 tx=0x7f2bd802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.118+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/1675913000 shutdown_connections 2024-04-26T20:22:54.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.118+0000 7f2bf4ac8640 1 --2- 172.21.15.37:0/1675913000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00b1480 0x7f2bf00ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.118+0000 7f2bf4ac8640 1 --2- 172.21.15.37:0/1675913000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf015f4f0 0x7f2bf006c620 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.118+0000 7f2bf4ac8640 1 --2- 172.21.15.37:0/1675913000 >> v2:172.21.15.134:3300/0 conn(0x7f2bf006c240 0x7f2bf006fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.118+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/1675913000 >> 172.21.15.37:0/1675913000 conn(0x7f2bf00b3b70 msgr2=0x7f2bf00b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.118+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/1675913000 shutdown_connections 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.118+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/1675913000 wait complete. 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.119+0000 7f2bf4ac8640 1 Processor -- start 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.119+0000 7f2bf4ac8640 1 -- start start 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.120+0000 7f2bf4ac8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf006c240 0x7f2bf01499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.120+0000 7f2bf4ac8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00b1480 0x7f2bf0160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.120+0000 7f2bf4ac8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf015f4f0 0x7f2bf015b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.120+0000 7f2bf4ac8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf00aee00 con 0x7f2bf015f4f0 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.120+0000 7f2bf4ac8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf00b6230 con 0x7f2bf006c240 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.120+0000 7f2bf4ac8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf00b0fc0 con 0x7f2bf00b1480 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.120+0000 7f2beed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf015f4f0 0x7f2bf015b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.120+0000 7f2beed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf015f4f0 0x7f2bf015b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36020/0 (socket says 172.21.15.37:36020) 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.120+0000 7f2beed9d640 1 -- 172.21.15.37:0/2199423080 learned_addr learned my addr 172.21.15.37:0/2199423080 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.120+0000 7f2bee59c640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf006c240 0x7f2bf01499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.120+0000 7f2be7fff640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00b1480 0x7f2bf0160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.121+0000 7f2beed9d640 1 -- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00b1480 msgr2=0x7f2bf0160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.121+0000 7f2beed9d640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00b1480 0x7f2bf0160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.121+0000 7f2beed9d640 1 -- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf006c240 msgr2=0x7f2bf01499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.121+0000 7f2beed9d640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf006c240 0x7f2bf01499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.121+0000 7f2beed9d640 1 -- 172.21.15.37:0/2199423080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf015fdb0 con 0x7f2bf015f4f0 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.121+0000 7f2beed9d640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf015f4f0 0x7f2bf015b620 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2be8007e30 tx=0x7f2be800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.121+0000 7f2be6ffd640 1 -- 172.21.15.37:0/2199423080 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8014620 con 0x7f2bf015f4f0 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.121+0000 7f2be6ffd640 1 -- 172.21.15.37:0/2199423080 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be8019070 con 0x7f2bf015f4f0 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.121+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/2199423080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf0160090 con 0x7f2bf015f4f0 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.121+0000 7f2be6ffd640 1 -- 172.21.15.37:0/2199423080 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8014a00 con 0x7f2bf015f4f0 2024-04-26T20:22:54.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.121+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/2199423080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf0160650 con 0x7f2bf015f4f0 2024-04-26T20:22:54.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.122+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/2199423080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf006a360 con 0x7f2bf015f4f0 2024-04-26T20:22:54.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.122+0000 7f2be6ffd640 1 -- 172.21.15.37:0/2199423080 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2be8014ba0 con 0x7f2bf015f4f0 2024-04-26T20:22:54.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.123+0000 7f2be6ffd640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd006c8f0 0x7f2bd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:54.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.123+0000 7f2be6ffd640 1 -- 172.21.15.37:0/2199423080 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2be808c2f0 con 0x7f2bf015f4f0 2024-04-26T20:22:54.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.123+0000 7f2bee59c640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd006c8f0 0x7f2bd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:54.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.124+0000 7f2bee59c640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd006c8f0 0x7f2bd006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2be000ab60 tx=0x7f2be0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:54.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.127+0000 7f2be6ffd640 1 -- 172.21.15.37:0/2199423080 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2be805a030 con 0x7f2bf015f4f0 2024-04-26T20:22:54.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.351+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/2199423080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2bf006e360 con 0x7f2bf015f4f0 2024-04-26T20:22:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.351+0000 7f2be6ffd640 1 -- 172.21.15.37:0/2199423080 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7f2be805e330 con 0x7f2bf015f4f0 2024-04-26T20:22:54.353 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd006c8f0 msgr2=0x7f2bd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd006c8f0 0x7f2bd006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2be000ab60 tx=0x7f2be0008040 comp rx=0 tx=0).stop 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/2199423080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf015f4f0 msgr2=0x7f2bf015b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf015f4f0 0x7f2bf015b620 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2be8007e30 tx=0x7f2be800e4b0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/2199423080 shutdown_connections 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bf015f4f0 0x7f2bf015b620 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd006c8f0 0x7f2bd006edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00b1480 0x7f2bf0160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 --2- 172.21.15.37:0/2199423080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf006c240 0x7f2bf01499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/2199423080 >> 172.21.15.37:0/2199423080 conn(0x7f2bf00b3b70 msgr2=0x7f2bf00d1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/2199423080 shutdown_connections 2024-04-26T20:22:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:54.353+0000 7f2bf4ac8640 1 -- 172.21.15.37:0/2199423080 wait complete. 2024-04-26T20:22:54.366 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T20:22:54.366 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:22:54.366 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi134.front.sepia.ceph.com 2024-04-26T20:22:54.366 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T20:22:54.366 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:22:54.366 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T20:22:54.366 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T20:22:54.367 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:22:54.367 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:22:54.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:54.367 DEBUG:teuthology.orchestra.run.smithi134:> ip addr 2024-04-26T20:22:54.386 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:22:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e6 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f0 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e7 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f1 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e8 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:22:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e9 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.21.15.134/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft 42700sec preferred_lft 42700sec 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::5526:381c:36db:a6d8/64 scope link noprefixroute 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0a:82:83:43:1c:3f brd ff:ff:ff:ff:ff:ff 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::205c:9aff:fee0:b7b/64 scope link 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether c2:13:60:38:f8:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::c013:60ff:fe38:f842/64 scope link 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:54.388 INFO:teuthology.orchestra.run.smithi134.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:22:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0a:82:83:43:1c:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:22:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::882:83ff:fe43:1c3f/64 scope link 2024-04-26T20:22:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:22:54.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:54.389 DEBUG:teuthology.orchestra.run.smithi134:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:22:54.455 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-04-26T20:22:54.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:54.457 DEBUG:teuthology.orchestra.run.smithi134:> route 2024-04-26T20:22:54.515 INFO:teuthology.orchestra.run.smithi134.stdout:Kernel IP routing table 2024-04-26T20:22:54.515 INFO:teuthology.orchestra.run.smithi134.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:22:54.516 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:22:54.516 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:22:54.516 INFO:teuthology.orchestra.run.smithi134.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:22:54.516 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:22:54.516 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-26T20:22:54.516 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:22:54.516 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:22:54.516 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:22:54.517 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-26T20:22:54.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:22:54.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:54.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:54.647 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-26T20:22:54.662 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T20:22:54.662 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:22:54.662 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:22:54.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:54.663 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set brx.0 up 2024-04-26T20:22:54.727 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:54.727 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T20:22:54.783 INFO:teuthology.orchestra.run.smithi134.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:22:54.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:22:54.783 DEBUG:teuthology.orchestra.run.smithi134:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:22:54.838 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-26T20:22:54.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:54.838 DEBUG:teuthology.orchestra.run.smithi134:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:54.920 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:54.920 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:54.928 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:22:54.928 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:22:54.928 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-04-26T20:22:54.928 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: trying mount with new device syntax: 1@d49b1c18-e1bf-49f2-acda-0904fa7c6684.cephfs=/ 2024-04-26T20:22:54.928 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.37:3300/172.21.15.134:3300/172.21.15.134:3301" will pass to kernel 2024-04-26T20:22:54.930 INFO:teuthology.orchestra.run.smithi134.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T20:22:54.930 INFO:teuthology.orchestra.run.smithi134.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:22:54.931 INFO:teuthology.orchestra.run.smithi134.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:22:54.931 INFO:teuthology.orchestra.run.smithi134.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:22:54.931 INFO:teuthology.orchestra.run.smithi134.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:22:54.931 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:22:54.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:54.931 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:22:55.001 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:22:55.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:22:55.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:22:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:22:55.097 INFO:teuthology.orchestra.run.smithi134.stdout:client4808 2024-04-26T20:22:55.098 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:22:55.098 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:55.113 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-26T20:22:55.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.230+0000 7fa48703f640 1 Processor -- start 2024-04-26T20:22:55.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.230+0000 7fa48703f640 1 -- start start 2024-04-26T20:22:55.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.230+0000 7fa48703f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa48015f5d0 0x7fa4800d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.230+0000 7fa48703f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4800d5640 0x7fa4800b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.230+0000 7fa48703f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4800b50f0 con 0x7fa4800d4010 2024-04-26T20:22:55.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.230+0000 7fa48703f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa4800b5420 con 0x7fa48015f5d0 2024-04-26T20:22:55.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.230+0000 7fa48703f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa4800d0a80 con 0x7fa4800d5640 2024-04-26T20:22:55.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.231+0000 7fa484ddb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa48015f5d0 0x7fa4800d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.231+0000 7fa484ddb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa48015f5d0 0x7fa4800d3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36024/0 (socket says 172.21.15.37:36024) 2024-04-26T20:22:55.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.231+0000 7fa484ddb640 1 -- 172.21.15.37:0/1343046768 learned_addr learned my addr 172.21.15.37:0/1343046768 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:55.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.231+0000 7fa47ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa4800d4010 0x7fa4800d4f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52632/0 (socket says 172.21.15.37:52632) 2024-04-26T20:22:55.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.231+0000 7fa4855dc640 1 --2- 172.21.15.37:0/1343046768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4800d5640 0x7fa4800b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.231+0000 7fa484ddb640 1 -- 172.21.15.37:0/1343046768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4800d5640 msgr2=0x7fa4800b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:55.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.231+0000 7fa484ddb640 1 --2- 172.21.15.37:0/1343046768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4800d5640 0x7fa4800b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.231+0000 7fa484ddb640 1 -- 172.21.15.37:0/1343046768 >> v1:172.21.15.37:6789/0 conn(0x7fa4800d4010 legacy=0x7fa4800d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:55.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.231+0000 7fa47f7fe640 1 -- 172.21.15.37:0/1343046768 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2049994819 0 0) 0x7fa4800b50f0 con 0x7fa4800d4010 2024-04-26T20:22:55.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.231+0000 7fa484ddb640 1 -- 172.21.15.37:0/1343046768 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4800d0d60 con 0x7fa48015f5d0 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.232+0000 7fa484ddb640 1 --2- 172.21.15.37:0/1343046768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48015f5d0 0x7fa4800d3ad0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa474009870 tx=0x7fa47402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6997189f57482f83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.232+0000 7fa47f7fe640 1 -- 172.21.15.37:0/1343046768 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474037070 con 0x7fa48015f5d0 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.232+0000 7fa47f7fe640 1 -- 172.21.15.37:0/1343046768 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47402ae50 con 0x7fa48015f5d0 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.232+0000 7fa47f7fe640 1 -- 172.21.15.37:0/1343046768 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474032860 con 0x7fa48015f5d0 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.232+0000 7fa48703f640 1 -- 172.21.15.37:0/1343046768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48015f5d0 msgr2=0x7fa4800d3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.232+0000 7fa48703f640 1 --2- 172.21.15.37:0/1343046768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48015f5d0 0x7fa4800d3ad0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa474009870 tx=0x7fa47402a300 comp rx=0 tx=0).stop 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.233+0000 7fa48703f640 1 -- 172.21.15.37:0/1343046768 shutdown_connections 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.233+0000 7fa48703f640 1 --2- 172.21.15.37:0/1343046768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4800d5640 0x7fa4800b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.233+0000 7fa48703f640 1 --2- 172.21.15.37:0/1343046768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48015f5d0 0x7fa4800d3ad0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.233+0000 7fa48703f640 1 -- 172.21.15.37:0/1343046768 >> 172.21.15.37:0/1343046768 conn(0x7fa48008ebd0 msgr2=0x7fa48009e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.233+0000 7fa48703f640 1 -- 172.21.15.37:0/1343046768 shutdown_connections 2024-04-26T20:22:55.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.233+0000 7fa48703f640 1 -- 172.21.15.37:0/1343046768 wait complete. 2024-04-26T20:22:55.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.234+0000 7fa48703f640 1 Processor -- start 2024-04-26T20:22:55.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.234+0000 7fa48703f640 1 -- start start 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa48703f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4800d4010 0x7fa4801504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa48703f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4800d5640 0x7fa480160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa48703f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48015f5d0 0x7fa4800b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa48703f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4800d34f0 con 0x7fa48015f5d0 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa48703f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa48015f9b0 con 0x7fa4800d5640 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa48703f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa4800b5df0 con 0x7fa4800d4010 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa4855dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48015f5d0 0x7fa4800b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa4855dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48015f5d0 0x7fa4800b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36036/0 (socket says 172.21.15.37:36036) 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa4855dc640 1 -- 172.21.15.37:0/2843281474 learned_addr learned my addr 172.21.15.37:0/2843281474 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa47ffff640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4800d5640 0x7fa480160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa484ddb640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4800d4010 0x7fa4801504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa4855dc640 1 -- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4800d4010 msgr2=0x7fa4801504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa4855dc640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4800d4010 0x7fa4801504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa4855dc640 1 -- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4800d5640 msgr2=0x7fa480160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa4855dc640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4800d5640 0x7fa480160d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa4855dc640 1 -- 172.21.15.37:0/2843281474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa480161790 con 0x7fa48015f5d0 2024-04-26T20:22:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa484ddb640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4800d4010 0x7fa4801504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa4855dc640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48015f5d0 0x7fa4800b2210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa47800efc0 tx=0x7fa47800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa47d7fa640 1 -- 172.21.15.37:0/2843281474 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478018020 con 0x7fa48015f5d0 2024-04-26T20:22:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.235+0000 7fa47d7fa640 1 -- 172.21.15.37:0/2843281474 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa478014480 con 0x7fa48015f5d0 2024-04-26T20:22:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.236+0000 7fa48703f640 1 -- 172.21.15.37:0/2843281474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa480161a70 con 0x7fa48015f5d0 2024-04-26T20:22:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.236+0000 7fa47d7fa640 1 -- 172.21.15.37:0/2843281474 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478014870 con 0x7fa48015f5d0 2024-04-26T20:22:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.236+0000 7fa48703f640 1 -- 172.21.15.37:0/2843281474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa480162030 con 0x7fa48015f5d0 2024-04-26T20:22:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.236+0000 7fa48703f640 1 -- 172.21.15.37:0/2843281474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48006a360 con 0x7fa48015f5d0 2024-04-26T20:22:55.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.237+0000 7fa47d7fa640 1 -- 172.21.15.37:0/2843281474 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa478005d10 con 0x7fa48015f5d0 2024-04-26T20:22:55.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.238+0000 7fa47d7fa640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa46406c8f0 0x7fa46406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.238+0000 7fa47d7fa640 1 -- 172.21.15.37:0/2843281474 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa47808cac0 con 0x7fa48015f5d0 2024-04-26T20:22:55.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.238+0000 7fa484ddb640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa46406c8f0 0x7fa46406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.238+0000 7fa484ddb640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa46406c8f0 0x7fa46406edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa474034000 tx=0x7fa47402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.242+0000 7fa47d7fa640 1 -- 172.21.15.37:0/2843281474 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa478022030 con 0x7fa48015f5d0 2024-04-26T20:22:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.467+0000 7fa48703f640 1 -- 172.21.15.37:0/2843281474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa4800b4160 con 0x7fa48015f5d0 2024-04-26T20:22:55.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.782+0000 7fa47d7fa640 1 -- 172.21.15.37:0/2843281474 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v16) ==== 136+0+0 (secure 0 0 0) 0x7fa478010040 con 0x7fa48015f5d0 2024-04-26T20:22:55.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.784+0000 7fa48703f640 1 -- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa46406c8f0 msgr2=0x7fa46406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:55.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.784+0000 7fa48703f640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa46406c8f0 0x7fa46406edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa474034000 tx=0x7fa47402e000 comp rx=0 tx=0).stop 2024-04-26T20:22:55.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.784+0000 7fa48703f640 1 -- 172.21.15.37:0/2843281474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48015f5d0 msgr2=0x7fa4800b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:55.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.784+0000 7fa48703f640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48015f5d0 0x7fa4800b2210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa47800efc0 tx=0x7fa47800c490 comp rx=0 tx=0).stop 2024-04-26T20:22:55.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.784+0000 7fa48703f640 1 -- 172.21.15.37:0/2843281474 shutdown_connections 2024-04-26T20:22:55.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.784+0000 7fa48703f640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48015f5d0 0x7fa4800b2210 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.784+0000 7fa48703f640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4800d5640 0x7fa480160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.784+0000 7fa48703f640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa46406c8f0 0x7fa46406edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.784+0000 7fa48703f640 1 --2- 172.21.15.37:0/2843281474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4800d4010 0x7fa4801504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.784+0000 7fa48703f640 1 -- 172.21.15.37:0/2843281474 >> 172.21.15.37:0/2843281474 conn(0x7fa48008ebd0 msgr2=0x7fa4800d4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.785+0000 7fa48703f640 1 -- 172.21.15.37:0/2843281474 shutdown_connections 2024-04-26T20:22:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.785+0000 7fa48703f640 1 -- 172.21.15.37:0/2843281474 wait complete. 2024-04-26T20:22:55.797 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-26T20:22:55.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.899+0000 7f403bac5640 1 Processor -- start 2024-04-26T20:22:55.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.899+0000 7f403bac5640 1 -- start start 2024-04-26T20:22:55.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f403bac5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4034162e40 0x7f40340d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f403bac5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340b2050 0x7f40340cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f403bac5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40340b1d70 con 0x7f40340cfd70 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f403bac5640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4034163220 con 0x7f4034162e40 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f403bac5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f40340d0eb0 con 0x7f40340b2050 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f4039861640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4034162e40 0x7f40340d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f4039861640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4034162e40 0x7f40340d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50698/0 (socket says 172.21.15.37:50698) 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f4039861640 1 -- 172.21.15.37:0/2376574566 learned_addr learned my addr 172.21.15.37:0/2376574566 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f4039060640 1 --2- 172.21.15.37:0/2376574566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340b2050 0x7f40340cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f4039861640 1 -- 172.21.15.37:0/2376574566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340b2050 msgr2=0x7f40340cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f4039861640 1 --2- 172.21.15.37:0/2376574566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340b2050 0x7f40340cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f4039861640 1 -- 172.21.15.37:0/2376574566 >> v1:172.21.15.134:6789/0 conn(0x7f40340cfd70 legacy=0x7f40340ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:55.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f4039861640 1 -- 172.21.15.37:0/2376574566 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40340d1960 con 0x7f4034162e40 2024-04-26T20:22:55.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.900+0000 7f4039060640 1 --2- 172.21.15.37:0/2376574566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340b2050 0x7f40340cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:55.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.901+0000 7f4039861640 1 --2- 172.21.15.37:0/2376574566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034162e40 0x7f40340d1340 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4030009870 tx=0x7f403002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b87229edbc925ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.901+0000 7f403885f640 1 -- 172.21.15.37:0/2376574566 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030037070 con 0x7f4034162e40 2024-04-26T20:22:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.901+0000 7f403885f640 1 -- 172.21.15.37:0/2376574566 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4030032440 con 0x7f4034162e40 2024-04-26T20:22:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.901+0000 7f403885f640 1 -- 172.21.15.37:0/2376574566 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030032810 con 0x7f4034162e40 2024-04-26T20:22:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.902+0000 7f403bac5640 1 -- 172.21.15.37:0/2376574566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034162e40 msgr2=0x7f40340d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.902+0000 7f403bac5640 1 --2- 172.21.15.37:0/2376574566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034162e40 0x7f40340d1340 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4030009870 tx=0x7f403002a1c0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.902+0000 7f403bac5640 1 -- 172.21.15.37:0/2376574566 shutdown_connections 2024-04-26T20:22:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.902+0000 7f403bac5640 1 --2- 172.21.15.37:0/2376574566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340b2050 0x7f40340cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.902+0000 7f403bac5640 1 --2- 172.21.15.37:0/2376574566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034162e40 0x7f40340d1340 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.902+0000 7f403bac5640 1 -- 172.21.15.37:0/2376574566 >> 172.21.15.37:0/2376574566 conn(0x7f403409ba50 msgr2=0x7f403409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:55.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.902+0000 7f403bac5640 1 -- 172.21.15.37:0/2376574566 shutdown_connections 2024-04-26T20:22:55.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.902+0000 7f403bac5640 1 -- 172.21.15.37:0/2376574566 wait complete. 2024-04-26T20:22:55.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.903+0000 7f403bac5640 1 Processor -- start 2024-04-26T20:22:55.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.903+0000 7f403bac5640 1 -- start start 2024-04-26T20:22:55.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403bac5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40340b2050 0x7f403414f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403bac5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340cfd70 0x7f40341f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403bac5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034162e40 0x7f403415f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403bac5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40340d1f20 con 0x7f4034162e40 2024-04-26T20:22:55.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403bac5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f40340d01a0 con 0x7f40340b2050 2024-04-26T20:22:55.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403bac5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f40340b24b0 con 0x7f40340cfd70 2024-04-26T20:22:55.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403a062640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034162e40 0x7f403415f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403a062640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034162e40 0x7f403415f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50706/0 (socket says 172.21.15.37:50706) 2024-04-26T20:22:55.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403a062640 1 -- 172.21.15.37:0/2377045448 learned_addr learned my addr 172.21.15.37:0/2377045448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:55.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f4039060640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340cfd70 0x7f40341f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f4039861640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40340b2050 0x7f403414f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403a062640 1 -- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340cfd70 msgr2=0x7f40341f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:55.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403a062640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340cfd70 0x7f40341f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403a062640 1 -- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40340b2050 msgr2=0x7f403414f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:55.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403a062640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40340b2050 0x7f403414f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403a062640 1 -- 172.21.15.37:0/2377045448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403415fe90 con 0x7f4034162e40 2024-04-26T20:22:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f4039060640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340cfd70 0x7f40341f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f4039861640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40340b2050 0x7f403414f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403a062640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034162e40 0x7f403415f950 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c7a0 tx=0x7f402c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f402a7fc640 1 -- 172.21.15.37:0/2377045448 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c003b70 con 0x7f4034162e40 2024-04-26T20:22:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f402a7fc640 1 -- 172.21.15.37:0/2377045448 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f402c003d30 con 0x7f4034162e40 2024-04-26T20:22:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f402a7fc640 1 -- 172.21.15.37:0/2377045448 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c00aa90 con 0x7f4034162e40 2024-04-26T20:22:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403bac5640 1 -- 172.21.15.37:0/2377045448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4034160130 con 0x7f4034162e40 2024-04-26T20:22:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.904+0000 7f403bac5640 1 -- 172.21.15.37:0/2377045448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40341606f0 con 0x7f4034162e40 2024-04-26T20:22:55.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.905+0000 7f403bac5640 1 -- 172.21.15.37:0/2377045448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40341608f0 con 0x7f4034162e40 2024-04-26T20:22:55.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.906+0000 7f402a7fc640 1 -- 172.21.15.37:0/2377045448 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f402c00ac30 con 0x7f4034162e40 2024-04-26T20:22:55.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.906+0000 7f402a7fc640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f401806c8f0 0x7f401806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:55.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.907+0000 7f402a7fc640 1 -- 172.21.15.37:0/2377045448 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f402c08ba20 con 0x7f4034162e40 2024-04-26T20:22:55.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.907+0000 7f4039861640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f401806c8f0 0x7f401806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.907+0000 7f4039861640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f401806c8f0 0x7f401806edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4030034000 tx=0x7f403002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:55.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:55.911+0000 7f402a7fc640 1 -- 172.21.15.37:0/2377045448 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40341608f0 con 0x7f4034162e40 2024-04-26T20:22:56.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.124+0000 7f403bac5640 1 -- 172.21.15.37:0/2377045448 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f403406a360 con 0x7f401806c8f0 2024-04-26T20:22:56.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.126+0000 7f402a7fc640 1 -- 172.21.15.37:0/2377045448 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f403406a360 con 0x7f401806c8f0 2024-04-26T20:22:56.127 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-26T20:22:56.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:22:56.127 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephfs" 2024-04-26T20:22:56.127 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:22:56.127 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-26T20:22:56.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.127+0000 7f403bac5640 1 -- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f401806c8f0 msgr2=0x7f401806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.127+0000 7f403bac5640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f401806c8f0 0x7f401806edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4030034000 tx=0x7f403002e000 comp rx=0 tx=0).stop 2024-04-26T20:22:56.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.127+0000 7f403bac5640 1 -- 172.21.15.37:0/2377045448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034162e40 msgr2=0x7f403415f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.127+0000 7f403bac5640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034162e40 0x7f403415f950 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c7a0 tx=0x7f402c00cc60 comp rx=0 tx=0).stop 2024-04-26T20:22:56.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.127+0000 7f403bac5640 1 -- 172.21.15.37:0/2377045448 shutdown_connections 2024-04-26T20:22:56.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.128+0000 7f403bac5640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034162e40 0x7f403415f950 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.128+0000 7f403bac5640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f401806c8f0 0x7f401806edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.128+0000 7f403bac5640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40340cfd70 0x7f40341f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.128+0000 7f403bac5640 1 --2- 172.21.15.37:0/2377045448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40340b2050 0x7f403414f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.128+0000 7f403bac5640 1 -- 172.21.15.37:0/2377045448 >> 172.21.15.37:0/2377045448 conn(0x7f403409ba50 msgr2=0x7f40340cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.128+0000 7f403bac5640 1 -- 172.21.15.37:0/2377045448 shutdown_connections 2024-04-26T20:22:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.128+0000 7f403bac5640 1 -- 172.21.15.37:0/2377045448 wait complete. 2024-04-26T20:22:56.139 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:22:56.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.244+0000 7f6c57389640 1 Processor -- start 2024-04-26T20:22:56.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.245+0000 7f6c57389640 1 -- start start 2024-04-26T20:22:56.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.245+0000 7f6c57389640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c500d2b30 0x7f6c500cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.245+0000 7f6c57389640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6c5006e470 0x7f6c500ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.245+0000 7f6c57389640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c500d25f0 con 0x7f6c5015f5e0 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.245+0000 7f6c57389640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6c500cda70 con 0x7f6c500d2b30 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.245+0000 7f6c57389640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6c500d2030 con 0x7f6c5006e470 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.245+0000 7f6c54924640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c500d2b30 0x7f6c500cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.245+0000 7f6c55125640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6c5015f5e0 0x7f6c5006dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50784/0 (socket says 172.21.15.37:50784) 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.245+0000 7f6c55125640 1 -- 172.21.15.37:0/1664371321 learned_addr learned my addr 172.21.15.37:0/1664371321 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.246+0000 7f6c54924640 1 -- 172.21.15.37:0/1664371321 >> v2:172.21.15.134:3300/0 conn(0x7f6c5006e470 msgr2=0x7f6c500ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.246+0000 7f6c55926640 1 --2- 172.21.15.37:0/1664371321 >> v2:172.21.15.134:3300/0 conn(0x7f6c5006e470 0x7f6c500ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.246+0000 7f6c54924640 1 --2- 172.21.15.37:0/1664371321 >> v2:172.21.15.134:3300/0 conn(0x7f6c5006e470 0x7f6c500ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.246+0000 7f6c54924640 1 -- 172.21.15.37:0/1664371321 >> v1:172.21.15.37:6789/0 conn(0x7f6c5015f5e0 legacy=0x7f6c5006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.246+0000 7f6c47fff640 1 -- 172.21.15.37:0/1664371321 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865595247 0 0) 0x7f6c500d25f0 con 0x7f6c5015f5e0 2024-04-26T20:22:56.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.246+0000 7f6c54924640 1 -- 172.21.15.37:0/1664371321 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c500d2310 con 0x7f6c500d2b30 2024-04-26T20:22:56.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.246+0000 7f6c55926640 1 --2- 172.21.15.37:0/1664371321 >> v2:172.21.15.134:3300/0 conn(0x7f6c5006e470 0x7f6c500ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:56.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.246+0000 7f6c54924640 1 --2- 172.21.15.37:0/1664371321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c500d2b30 0x7f6c500cdf00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6c5006cd00 tx=0x7f6c4002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a297858089396fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:56.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.247+0000 7f6c47fff640 1 -- 172.21.15.37:0/1664371321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40037070 con 0x7f6c500d2b30 2024-04-26T20:22:56.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.247+0000 7f6c47fff640 1 -- 172.21.15.37:0/1664371321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c40032440 con 0x7f6c500d2b30 2024-04-26T20:22:56.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.247+0000 7f6c47fff640 1 -- 172.21.15.37:0/1664371321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4002ae60 con 0x7f6c500d2b30 2024-04-26T20:22:56.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.247+0000 7f6c57389640 1 -- 172.21.15.37:0/1664371321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c500d2b30 msgr2=0x7f6c500cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.247+0000 7f6c57389640 1 --2- 172.21.15.37:0/1664371321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c500d2b30 0x7f6c500cdf00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6c5006cd00 tx=0x7f6c4002a110 comp rx=0 tx=0).stop 2024-04-26T20:22:56.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.247+0000 7f6c57389640 1 -- 172.21.15.37:0/1664371321 shutdown_connections 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.247+0000 7f6c57389640 1 --2- 172.21.15.37:0/1664371321 >> v2:172.21.15.134:3300/0 conn(0x7f6c5006e470 0x7f6c500ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.247+0000 7f6c57389640 1 --2- 172.21.15.37:0/1664371321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c500d2b30 0x7f6c500cdf00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.247+0000 7f6c57389640 1 -- 172.21.15.37:0/1664371321 >> 172.21.15.37:0/1664371321 conn(0x7f6c500d2f80 msgr2=0x7f6c500d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.247+0000 7f6c57389640 1 -- 172.21.15.37:0/1664371321 shutdown_connections 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.247+0000 7f6c57389640 1 -- 172.21.15.37:0/1664371321 wait complete. 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.248+0000 7f6c57389640 1 Processor -- start 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.249+0000 7f6c57389640 1 -- start start 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.249+0000 7f6c57389640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c5006e470 0x7f6c501f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.249+0000 7f6c57389640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c500d2b30 0x7f6c500cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.249+0000 7f6c57389640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5015f5e0 0x7f6c50161bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.249+0000 7f6c57389640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5006d560 con 0x7f6c500d2b30 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.249+0000 7f6c57389640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500d1cb0 con 0x7f6c5006e470 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.249+0000 7f6c57389640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5006d6e0 con 0x7f6c5015f5e0 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.249+0000 7f6c54924640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c500d2b30 0x7f6c500cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.249+0000 7f6c54924640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c500d2b30 0x7f6c500cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50730/0 (socket says 172.21.15.37:50730) 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.249+0000 7f6c54924640 1 -- 172.21.15.37:0/1552085516 learned_addr learned my addr 172.21.15.37:0/1552085516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c55125640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c5006e470 0x7f6c501f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c54924640 1 -- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5015f5e0 msgr2=0x7f6c50161bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c55926640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5015f5e0 0x7f6c50161bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c54924640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5015f5e0 0x7f6c50161bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c54924640 1 -- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c5006e470 msgr2=0x7f6c501f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c54924640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c5006e470 0x7f6c501f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c54924640 1 -- 172.21.15.37:0/1552085516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c501f5da0 con 0x7f6c500d2b30 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c54924640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c500d2b30 0x7f6c500cb3a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6c40000c00 tx=0x7f6c400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c45ffb640 1 -- 172.21.15.37:0/1552085516 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40037040 con 0x7f6c500d2b30 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c45ffb640 1 -- 172.21.15.37:0/1552085516 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c40042070 con 0x7f6c500d2b30 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c57389640 1 -- 172.21.15.37:0/1552085516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5015fdd0 con 0x7f6c500d2b30 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c45ffb640 1 -- 172.21.15.37:0/1552085516 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40030a50 con 0x7f6c500d2b30 2024-04-26T20:22:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.250+0000 7f6c57389640 1 -- 172.21.15.37:0/1552085516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c50160390 con 0x7f6c500d2b30 2024-04-26T20:22:56.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.251+0000 7f6c57389640 1 -- 172.21.15.37:0/1552085516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c500693e0 con 0x7f6c500d2b30 2024-04-26T20:22:56.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.252+0000 7f6c45ffb640 1 -- 172.21.15.37:0/1552085516 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c40030bf0 con 0x7f6c500d2b30 2024-04-26T20:22:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.252+0000 7f6c45ffb640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c3406c8f0 0x7f6c3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.252+0000 7f6c45ffb640 1 -- 172.21.15.37:0/1552085516 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6c400ac5e0 con 0x7f6c500d2b30 2024-04-26T20:22:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.252+0000 7f6c55125640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c3406c8f0 0x7f6c3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.253+0000 7f6c55125640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c3406c8f0 0x7f6c3406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c000f60 tx=0x7f6c4c007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:56.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.256+0000 7f6c45ffb640 1 -- 172.21.15.37:0/1552085516 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6c4007aa10 con 0x7f6c500d2b30 2024-04-26T20:22:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.443+0000 7f6c57389640 1 -- 172.21.15.37:0/1552085516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6c5006df00 con 0x7f6c500d2b30 2024-04-26T20:22:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.443+0000 7f6c45ffb640 1 -- 172.21.15.37:0/1552085516 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f6c4007a310 con 0x7f6c500d2b30 2024-04-26T20:22:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 -- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c3406c8f0 msgr2=0x7f6c3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c3406c8f0 0x7f6c3406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c000f60 tx=0x7f6c4c007c00 comp rx=0 tx=0).stop 2024-04-26T20:22:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 -- 172.21.15.37:0/1552085516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c500d2b30 msgr2=0x7f6c500cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c500d2b30 0x7f6c500cb3a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6c40000c00 tx=0x7f6c400046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 -- 172.21.15.37:0/1552085516 shutdown_connections 2024-04-26T20:22:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6c3406c8f0 0x7f6c3406edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5015f5e0 0x7f6c50161bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c500d2b30 0x7f6c500cb3a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 --2- 172.21.15.37:0/1552085516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c5006e470 0x7f6c501f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 -- 172.21.15.37:0/1552085516 >> 172.21.15.37:0/1552085516 conn(0x7f6c500d2f80 msgr2=0x7f6c500c8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 -- 172.21.15.37:0/1552085516 shutdown_connections 2024-04-26T20:22:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.445+0000 7f6c57389640 1 -- 172.21.15.37:0/1552085516 wait complete. 2024-04-26T20:22:56.457 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol19350 --mode=777 2024-04-26T20:22:56.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.560+0000 7fcb70437640 1 Processor -- start 2024-04-26T20:22:56.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.560+0000 7fcb70437640 1 -- start start 2024-04-26T20:22:56.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.561+0000 7fcb70437640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcb6815f310 0x7fcb68163850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.561+0000 7fcb70437640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb6815f6f0 con 0x7fcb680d2ab0 2024-04-26T20:22:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.561+0000 7fcb70437640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb68163d90 con 0x7fcb680b51f0 2024-04-26T20:22:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.561+0000 7fcb70437640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fcb680d0bb0 con 0x7fcb6815f310 2024-04-26T20:22:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.561+0000 7fcb6e1d3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcb6815f310 0x7fcb68163850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.561+0000 7fcb6d9d2640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fcb680d2ab0 0x7fcb680b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50792/0 (socket says 172.21.15.37:50792) 2024-04-26T20:22:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.561+0000 7fcb6d9d2640 1 -- 172.21.15.37:0/822290348 learned_addr learned my addr 172.21.15.37:0/822290348 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:22:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.561+0000 7fcb6e1d3640 1 -- 172.21.15.37:0/822290348 >> v1:172.21.15.134:6789/0 conn(0x7fcb680b51f0 legacy=0x7fcb680b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.561+0000 7fcb6e1d3640 1 -- 172.21.15.37:0/822290348 >> v1:172.21.15.37:6789/0 conn(0x7fcb680d2ab0 legacy=0x7fcb680b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.561+0000 7fcb6e1d3640 1 -- 172.21.15.37:0/822290348 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6806ef20 con 0x7fcb6815f310 2024-04-26T20:22:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.561+0000 7fcb6d1d1640 1 -- 172.21.15.37:0/822290348 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542440982 0 0) 0x7fcb6815f6f0 con 0x7fcb680d2ab0 2024-04-26T20:22:56.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.562+0000 7fcb6e1d3640 1 --2- 172.21.15.37:0/822290348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb6815f310 0x7fcb68163850 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcb600097a0 tx=0x7fcb6002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9e5acf0e3f2ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:56.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.562+0000 7fcb6d1d1640 1 -- 172.21.15.37:0/822290348 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60037070 con 0x7fcb6815f310 2024-04-26T20:22:56.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.562+0000 7fcb6d1d1640 1 -- 172.21.15.37:0/822290348 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb60032440 con 0x7fcb6815f310 2024-04-26T20:22:56.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.562+0000 7fcb6d1d1640 1 -- 172.21.15.37:0/822290348 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60032810 con 0x7fcb6815f310 2024-04-26T20:22:56.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.563+0000 7fcb70437640 1 -- 172.21.15.37:0/822290348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb6815f310 msgr2=0x7fcb68163850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.563+0000 7fcb70437640 1 --2- 172.21.15.37:0/822290348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb6815f310 0x7fcb68163850 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcb600097a0 tx=0x7fcb6002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.563+0000 7fcb70437640 1 -- 172.21.15.37:0/822290348 shutdown_connections 2024-04-26T20:22:56.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.563+0000 7fcb70437640 1 --2- 172.21.15.37:0/822290348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb6815f310 0x7fcb68163850 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.563+0000 7fcb70437640 1 -- 172.21.15.37:0/822290348 >> 172.21.15.37:0/822290348 conn(0x7fcb68071cf0 msgr2=0x7fcb680720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:56.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.563+0000 7fcb70437640 1 -- 172.21.15.37:0/822290348 shutdown_connections 2024-04-26T20:22:56.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.563+0000 7fcb70437640 1 -- 172.21.15.37:0/822290348 wait complete. 2024-04-26T20:22:56.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.564+0000 7fcb70437640 1 Processor -- start 2024-04-26T20:22:56.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.564+0000 7fcb70437640 1 -- start start 2024-04-26T20:22:56.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb70437640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb680b51f0 0x7fcb681f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb70437640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb680d2ab0 0x7fcb6815fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb70437640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb6815f310 0x7fcb68159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb70437640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb680d10e0 con 0x7fcb680b51f0 2024-04-26T20:22:56.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb70437640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb680d0e00 con 0x7fcb6815f310 2024-04-26T20:22:56.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb70437640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcb680b66d0 con 0x7fcb680d2ab0 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb6e1d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb680b51f0 0x7fcb681f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb6e1d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb680b51f0 0x7fcb681f94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50742/0 (socket says 172.21.15.37:50742) 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb6e1d3640 1 -- 172.21.15.37:0/105589080 learned_addr learned my addr 172.21.15.37:0/105589080 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb6e1d3640 1 -- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb680d2ab0 msgr2=0x7fcb6815fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb6e1d3640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb680d2ab0 0x7fcb6815fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb6e1d3640 1 -- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb6815f310 msgr2=0x7fcb68159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb6e1d3640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb6815f310 0x7fcb68159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb6e1d3640 1 -- 172.21.15.37:0/105589080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb681605c0 con 0x7fcb680b51f0 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.565+0000 7fcb6e1d3640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb680b51f0 0x7fcb681f94a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcb600097a0 tx=0x7fcb600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.566+0000 7fcb66ffd640 1 -- 172.21.15.37:0/105589080 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60037050 con 0x7fcb680b51f0 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.566+0000 7fcb66ffd640 1 -- 172.21.15.37:0/105589080 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb60038040 con 0x7fcb680b51f0 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.566+0000 7fcb70437640 1 -- 172.21.15.37:0/105589080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb681608a0 con 0x7fcb680b51f0 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.566+0000 7fcb66ffd640 1 -- 172.21.15.37:0/105589080 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60004470 con 0x7fcb680b51f0 2024-04-26T20:22:56.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.566+0000 7fcb70437640 1 -- 172.21.15.37:0/105589080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb68160e60 con 0x7fcb680b51f0 2024-04-26T20:22:56.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.567+0000 7fcb70437640 1 -- 172.21.15.37:0/105589080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb6806a360 con 0x7fcb680b51f0 2024-04-26T20:22:56.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.567+0000 7fcb66ffd640 1 -- 172.21.15.37:0/105589080 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb60030900 con 0x7fcb680b51f0 2024-04-26T20:22:56.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.568+0000 7fcb66ffd640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb5406c920 0x7fcb5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.568+0000 7fcb66ffd640 1 -- 172.21.15.37:0/105589080 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fcb600ac7f0 con 0x7fcb680b51f0 2024-04-26T20:22:56.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.568+0000 7fcb6d9d2640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb5406c920 0x7fcb5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.568+0000 7fcb6d9d2640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb5406c920 0x7fcb5406ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c007fd0 tx=0x7fcb5c007900 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:56.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.572+0000 7fcb66ffd640 1 -- 172.21.15.37:0/105589080 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb6007ab80 con 0x7fcb680b51f0 2024-04-26T20:22:56.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.785+0000 7fcb70437640 1 -- 172.21.15.37:0/105589080 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol19350", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fcb68160360 con 0x7fcb5406c920 2024-04-26T20:22:56.791 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:22:56.790+0000 7f9a851f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:22:56.791 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:22:56.790+0000 7f9a851f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:22:56.791 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:22:56.790+0000 7f9a851f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:22:56.791 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:22:56.790+0000 7f9a851f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:22:56.791 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:22:56.790+0000 7f9a851f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:22:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.826+0000 7fcb66ffd640 1 -- 172.21.15.37:0/105589080 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb5c0070d0 con 0x7fcb5406c920 2024-04-26T20:22:56.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 -- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb5406c920 msgr2=0x7fcb5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb5406c920 0x7fcb5406ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c007fd0 tx=0x7fcb5c007900 comp rx=0 tx=0).stop 2024-04-26T20:22:56.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 -- 172.21.15.37:0/105589080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb680b51f0 msgr2=0x7fcb681f94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb680b51f0 0x7fcb681f94a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcb600097a0 tx=0x7fcb600046c0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 -- 172.21.15.37:0/105589080 shutdown_connections 2024-04-26T20:22:56.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb6815f310 0x7fcb68159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcb5406c920 0x7fcb5406ede0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb680d2ab0 0x7fcb6815fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 --2- 172.21.15.37:0/105589080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb680b51f0 0x7fcb681f94a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 -- 172.21.15.37:0/105589080 >> 172.21.15.37:0/105589080 conn(0x7fcb68071cf0 msgr2=0x7fcb680b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:56.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 -- 172.21.15.37:0/105589080 shutdown_connections 2024-04-26T20:22:56.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.828+0000 7fcb70437640 1 -- 172.21.15.37:0/105589080 wait complete. 2024-04-26T20:22:56.840 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol19350 2024-04-26T20:22:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.942+0000 7fb5fd42a640 1 Processor -- start 2024-04-26T20:22:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.942+0000 7fb5fd42a640 1 -- start start 2024-04-26T20:22:56.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5fd42a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f815cdf0 0x7fb5f80d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5fd42a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb5f815d1d0 0x7fb5f80b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5fd42a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5f80d2590 con 0x7fb5f80d32d0 2024-04-26T20:22:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5fd42a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f80d4310 con 0x7fb5f815d1d0 2024-04-26T20:22:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5fd42a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f80d2870 con 0x7fb5f815cdf0 2024-04-26T20:22:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5f77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb5f815d1d0 0x7fb5f80b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5f77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb5f815d1d0 0x7fb5f80b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50758/0 (socket says 172.21.15.37:50758) 2024-04-26T20:22:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5f77fe640 1 -- 172.21.15.37:0/2397481137 learned_addr learned my addr 172.21.15.37:0/2397481137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5f6ffd640 1 --2- 172.21.15.37:0/2397481137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f815cdf0 0x7fb5f80d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5f77fe640 1 -- 172.21.15.37:0/2397481137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f815cdf0 msgr2=0x7fb5f80d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5f77fe640 1 --2- 172.21.15.37:0/2397481137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f815cdf0 0x7fb5f80d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5f77fe640 1 -- 172.21.15.37:0/2397481137 >> v1:172.21.15.134:6789/0 conn(0x7fb5f80d32d0 legacy=0x7fb5f80d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.943+0000 7fb5f77fe640 1 -- 172.21.15.37:0/2397481137 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f80b50b0 con 0x7fb5f815d1d0 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.944+0000 7fb5f77fe640 1 --2- 172.21.15.37:0/2397481137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f815d1d0 0x7fb5f80b58b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009e50 tx=0x7fb5f002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5171d9fcd17d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.944+0000 7fb5f5ffb640 1 -- 172.21.15.37:0/2397481137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0036070 con 0x7fb5f815d1d0 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.944+0000 7fb5f5ffb640 1 -- 172.21.15.37:0/2397481137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f0031600 con 0x7fb5f815d1d0 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.944+0000 7fb5f5ffb640 1 -- 172.21.15.37:0/2397481137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00319d0 con 0x7fb5f815d1d0 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.945+0000 7fb5fd42a640 1 -- 172.21.15.37:0/2397481137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f815d1d0 msgr2=0x7fb5f80b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.945+0000 7fb5fd42a640 1 --2- 172.21.15.37:0/2397481137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f815d1d0 0x7fb5f80b58b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009e50 tx=0x7fb5f002a710 comp rx=0 tx=0).stop 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.945+0000 7fb5fd42a640 1 -- 172.21.15.37:0/2397481137 shutdown_connections 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.945+0000 7fb5fd42a640 1 --2- 172.21.15.37:0/2397481137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f815d1d0 0x7fb5f80b58b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.945+0000 7fb5fd42a640 1 --2- 172.21.15.37:0/2397481137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f815cdf0 0x7fb5f80d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.945+0000 7fb5fd42a640 1 -- 172.21.15.37:0/2397481137 >> 172.21.15.37:0/2397481137 conn(0x7fb5f808ec30 msgr2=0x7fb5f809e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.945+0000 7fb5fd42a640 1 -- 172.21.15.37:0/2397481137 shutdown_connections 2024-04-26T20:22:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.945+0000 7fb5fd42a640 1 -- 172.21.15.37:0/2397481137 wait complete. 2024-04-26T20:22:56.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.946+0000 7fb5fd42a640 1 Processor -- start 2024-04-26T20:22:56.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.946+0000 7fb5fd42a640 1 -- start start 2024-04-26T20:22:56.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5fd42a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f80d32d0 0x7fb5f80cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5fd42a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f815cdf0 0x7fb5f8161e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5fd42a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f815d1d0 0x7fb5f80cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5fd42a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f80d21e0 con 0x7fb5f815cdf0 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5fd42a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f815fcc0 con 0x7fb5f80d32d0 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5fd42a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f80b3b40 con 0x7fb5f815d1d0 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5f67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f815cdf0 0x7fb5f8161e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5f67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f815cdf0 0x7fb5f8161e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50762/0 (socket says 172.21.15.37:50762) 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5f67fc640 1 -- 172.21.15.37:0/3122750589 learned_addr learned my addr 172.21.15.37:0/3122750589 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5f6ffd640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f80d32d0 0x7fb5f80cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5f77fe640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f815d1d0 0x7fb5f80cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5f67fc640 1 -- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f815d1d0 msgr2=0x7fb5f80cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5f67fc640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f815d1d0 0x7fb5f80cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5f67fc640 1 -- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f80d32d0 msgr2=0x7fb5f80cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5f67fc640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f80d32d0 0x7fb5f80cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.947+0000 7fb5f67fc640 1 -- 172.21.15.37:0/3122750589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f80ce080 con 0x7fb5f815cdf0 2024-04-26T20:22:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.948+0000 7fb5f67fc640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f815cdf0 0x7fb5f8161e60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00c440 tx=0x7fb5ec001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:56.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.948+0000 7fb5df7fe640 1 -- 172.21.15.37:0/3122750589 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0052d0 con 0x7fb5f815cdf0 2024-04-26T20:22:56.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.948+0000 7fb5df7fe640 1 -- 172.21.15.37:0/3122750589 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5ec005470 con 0x7fb5f815cdf0 2024-04-26T20:22:56.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.948+0000 7fb5df7fe640 1 -- 172.21.15.37:0/3122750589 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec007260 con 0x7fb5f815cdf0 2024-04-26T20:22:56.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.948+0000 7fb5fd42a640 1 -- 172.21.15.37:0/3122750589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f80ce360 con 0x7fb5f815cdf0 2024-04-26T20:22:56.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.948+0000 7fb5fd42a640 1 -- 172.21.15.37:0/3122750589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f80ce920 con 0x7fb5f815cdf0 2024-04-26T20:22:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.949+0000 7fb5fd42a640 1 -- 172.21.15.37:0/3122750589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f806a360 con 0x7fb5f815cdf0 2024-04-26T20:22:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.949+0000 7fb5df7fe640 1 -- 172.21.15.37:0/3122750589 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5ec007400 con 0x7fb5f815cdf0 2024-04-26T20:22:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.950+0000 7fb5df7fe640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5e406c8f0 0x7fb5e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.950+0000 7fb5df7fe640 1 -- 172.21.15.37:0/3122750589 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb5ec08b010 con 0x7fb5f815cdf0 2024-04-26T20:22:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.950+0000 7fb5f6ffd640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5e406c8f0 0x7fb5e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.950+0000 7fb5f6ffd640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5e406c8f0 0x7fb5e406edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8006fd0 tx=0x7fb5e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:56.954+0000 7fb5df7fe640 1 -- 172.21.15.37:0/3122750589 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5ec0594d0 con 0x7fb5f815cdf0 2024-04-26T20:22:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.169+0000 7fb5fd42a640 1 -- 172.21.15.37:0/3122750589 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol19350", "target": ["mon-mgr", ""]}) -- 0x7fb5f80b4cf0 con 0x7fb5e406c8f0 2024-04-26T20:22:57.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.174+0000 7fb5df7fe640 1 -- 172.21.15.37:0/3122750589 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7fb5f80b4cf0 con 0x7fb5e406c8f0 2024-04-26T20:22:57.175 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85 2024-04-26T20:22:57.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 -- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5e406c8f0 msgr2=0x7fb5e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:57.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5e406c8f0 0x7fb5e406edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8006fd0 tx=0x7fb5e8008040 comp rx=0 tx=0).stop 2024-04-26T20:22:57.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 -- 172.21.15.37:0/3122750589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f815cdf0 msgr2=0x7fb5f8161e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:57.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f815cdf0 0x7fb5f8161e60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00c440 tx=0x7fb5ec001e70 comp rx=0 tx=0).stop 2024-04-26T20:22:57.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 -- 172.21.15.37:0/3122750589 shutdown_connections 2024-04-26T20:22:57.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb5e406c8f0 0x7fb5e406edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:57.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f815d1d0 0x7fb5f80cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:57.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f815cdf0 0x7fb5f8161e60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:57.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 --2- 172.21.15.37:0/3122750589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f80d32d0 0x7fb5f80cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:57.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 -- 172.21.15.37:0/3122750589 >> 172.21.15.37:0/3122750589 conn(0x7fb5f808ec30 msgr2=0x7fb5f80d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:57.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 -- 172.21.15.37:0/3122750589 shutdown_connections 2024-04-26T20:22:57.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:22:57.176+0000 7fb5fd42a640 1 -- 172.21.15.37:0/3122750589 wait complete. 2024-04-26T20:22:57.188 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol19350 with 200 files each 1MB size under directory volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85 2024-04-26T20:22:57.188 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:57.188 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:57.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:57.204 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109787 s, 95.5 MB/s 2024-04-26T20:22:57.278 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:57.278 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:57.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:57.334 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:57.398 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:57.398 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:57.398 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892219 s, 118 MB/s 2024-04-26T20:22:57.399 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:57.399 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:57.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:57.456 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:57.523 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111852 s, 93.7 MB/s 2024-04-26T20:22:57.524 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:57.524 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:57.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:57.580 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:57.643 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:57.644 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:57.644 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723276 s, 145 MB/s 2024-04-26T20:22:57.644 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:57.644 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:57.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:57.699 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909068 s, 115 MB/s 2024-04-26T20:22:57.765 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:57.766 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:57.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:57.822 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715731 s, 147 MB/s 2024-04-26T20:22:57.886 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:57.886 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:57.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:57.942 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:58.005 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:58.005 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:58.005 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779641 s, 134 MB/s 2024-04-26T20:22:58.006 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:58.006 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:58.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:58.062 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:58.126 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:58.127 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:58.127 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920147 s, 114 MB/s 2024-04-26T20:22:58.127 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:58.127 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:58.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:58.183 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933328 s, 112 MB/s 2024-04-26T20:22:58.251 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:58.251 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:58.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:58.307 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:58.369 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:58.369 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:58.369 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724403 s, 145 MB/s 2024-04-26T20:22:58.370 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:58.370 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:58.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:58.427 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.10 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991676 s, 106 MB/s 2024-04-26T20:22:58.493 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:58.494 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:58.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:58.549 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.11 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:58.611 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:58.612 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:58.612 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749267 s, 140 MB/s 2024-04-26T20:22:58.613 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:58.613 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:58.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:58.668 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.12 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785394 s, 134 MB/s 2024-04-26T20:22:58.733 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:58.733 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:58.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:58.789 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.13 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:58.852 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:58.852 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:58.852 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779714 s, 134 MB/s 2024-04-26T20:22:58.853 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:58.853 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:58.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:58.910 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.14 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:58.975 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:58.975 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:58.975 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090336 s, 116 MB/s 2024-04-26T20:22:58.976 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:58.976 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:59.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:59.031 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.15 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:59.096 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:59.097 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:59.097 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974366 s, 108 MB/s 2024-04-26T20:22:59.098 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:59.098 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:59.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:59.153 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.16 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819593 s, 128 MB/s 2024-04-26T20:22:59.218 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:59.218 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:59.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:59.276 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.17 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:59.340 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:59.340 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:59.340 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816627 s, 128 MB/s 2024-04-26T20:22:59.340 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:59.340 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:59.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:59.400 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.18 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823219 s, 127 MB/s 2024-04-26T20:22:59.468 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:59.468 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:59.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:59.525 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.19 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100125 s, 105 MB/s 2024-04-26T20:22:59.592 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:59.592 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:59.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:59.662 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.20 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924741 s, 113 MB/s 2024-04-26T20:22:59.735 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:59.735 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:59.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:59.796 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.21 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:22:59.871 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:22:59.871 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:22:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120804 s, 86.8 MB/s 2024-04-26T20:22:59.872 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:22:59.872 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:22:59.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:22:59.933 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.22 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812866 s, 129 MB/s 2024-04-26T20:23:00.001 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:00.001 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:00.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:00.061 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.23 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907841 s, 116 MB/s 2024-04-26T20:23:00.130 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:00.130 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:00.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:00.190 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.24 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:00.257 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808235 s, 130 MB/s 2024-04-26T20:23:00.258 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:00.258 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:00.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:00.316 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.25 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954254 s, 110 MB/s 2024-04-26T20:23:00.386 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:00.386 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:00.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:00.443 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.26 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894597 s, 117 MB/s 2024-04-26T20:23:00.509 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:00.509 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:00.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:00.565 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.27 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:00.629 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:00.629 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:00.630 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900535 s, 116 MB/s 2024-04-26T20:23:00.631 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:00.631 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:00.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:00.687 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.28 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007796 s, 135 MB/s 2024-04-26T20:23:00.752 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:00.752 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:00.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:00.808 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.29 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:00.872 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:00.872 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:00.872 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739514 s, 142 MB/s 2024-04-26T20:23:00.872 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:00.872 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:00.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:00.928 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.30 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:00.990 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:00.990 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:00.990 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713297 s, 147 MB/s 2024-04-26T20:23:00.991 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:00.991 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:01.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:01.049 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.31 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791613 s, 132 MB/s 2024-04-26T20:23:01.113 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:01.113 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:01.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:01.169 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.32 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777382 s, 135 MB/s 2024-04-26T20:23:01.233 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:01.233 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:01.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:01.290 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.33 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945859 s, 111 MB/s 2024-04-26T20:23:01.356 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:01.356 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:01.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:01.412 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.34 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:01.476 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:01.476 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:01.476 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795169 s, 132 MB/s 2024-04-26T20:23:01.477 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:01.477 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:01.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:01.533 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.35 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:01.596 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:01.596 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:01.596 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754277 s, 139 MB/s 2024-04-26T20:23:01.597 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:01.597 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:01.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:01.654 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.36 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:01.717 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:01.718 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:01.718 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787742 s, 133 MB/s 2024-04-26T20:23:01.718 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:01.718 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:01.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:01.773 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.37 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:01.839 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:01.839 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:01.839 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795045 s, 132 MB/s 2024-04-26T20:23:01.839 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:01.839 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:01.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:01.894 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.38 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796063 s, 132 MB/s 2024-04-26T20:23:01.959 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:01.959 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:02.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:02.015 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.39 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:02.078 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:02.079 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:02.079 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758979 s, 138 MB/s 2024-04-26T20:23:02.079 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:02.079 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:02.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:02.135 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.40 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:02.200 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:02.201 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:02.201 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936554 s, 112 MB/s 2024-04-26T20:23:02.201 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:02.201 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:02.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:02.256 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.41 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:02.320 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:02.320 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:02.320 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750295 s, 140 MB/s 2024-04-26T20:23:02.321 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:02.321 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:02.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:02.377 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.42 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:02.441 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:02.441 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:02.441 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803715 s, 130 MB/s 2024-04-26T20:23:02.441 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:02.441 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:02.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:02.497 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.43 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755716 s, 139 MB/s 2024-04-26T20:23:02.561 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:02.561 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:02.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:02.617 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.44 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:02.681 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:02.681 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:02.681 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829134 s, 126 MB/s 2024-04-26T20:23:02.682 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:02.682 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:02.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:02.738 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.45 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:02.804 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:02.804 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:02.804 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104891 s, 100 MB/s 2024-04-26T20:23:02.805 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:02.805 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:02.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:02.861 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.46 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:02.924 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:02.924 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:02.924 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742876 s, 141 MB/s 2024-04-26T20:23:02.925 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:02.925 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:02.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:02.981 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.47 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:03.046 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:03.046 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:03.046 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787113 s, 133 MB/s 2024-04-26T20:23:03.047 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:03.047 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:03.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:03.102 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.48 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788003 s, 133 MB/s 2024-04-26T20:23:03.168 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:03.168 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:03.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:03.224 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.49 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777117 s, 135 MB/s 2024-04-26T20:23:03.291 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:03.291 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:03.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:03.346 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.50 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:03.411 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:03.411 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:03.412 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804025 s, 130 MB/s 2024-04-26T20:23:03.412 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:03.412 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:03.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:03.468 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.51 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:03.534 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:03.534 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:03.534 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922864 s, 114 MB/s 2024-04-26T20:23:03.534 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:03.534 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:03.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:03.593 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.52 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:03.656 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:03.656 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:03.656 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770318 s, 136 MB/s 2024-04-26T20:23:03.657 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:03.657 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:03.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:03.713 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.53 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063016 s, 166 MB/s 2024-04-26T20:23:03.778 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:03.778 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:03.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:03.833 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.54 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:03.897 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:03.897 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:03.897 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805198 s, 130 MB/s 2024-04-26T20:23:03.898 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:03.898 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:03.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:03.954 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.55 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:04.021 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:04.021 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:04.021 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108493 s, 96.6 MB/s 2024-04-26T20:23:04.021 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:04.021 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:04.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:04.077 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.56 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:04.141 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:04.141 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:04.141 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776503 s, 135 MB/s 2024-04-26T20:23:04.141 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:04.141 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:04.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:04.197 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.57 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:04.260 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:04.260 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:04.260 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765902 s, 137 MB/s 2024-04-26T20:23:04.261 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:04.261 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:04.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:04.317 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.58 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:04.384 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:04.384 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:04.384 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106657 s, 98.3 MB/s 2024-04-26T20:23:04.385 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:04.385 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:04.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:04.442 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.59 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:04.505 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:04.505 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:04.505 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781467 s, 134 MB/s 2024-04-26T20:23:04.506 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:04.506 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:04.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:04.561 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.60 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793486 s, 132 MB/s 2024-04-26T20:23:04.626 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:04.626 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:04.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:04.683 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.61 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:04.747 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:04.747 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:04.748 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956671 s, 110 MB/s 2024-04-26T20:23:04.748 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:04.749 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:04.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:04.805 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.62 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:04.871 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:04.871 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:04.871 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103912 s, 101 MB/s 2024-04-26T20:23:04.872 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:04.872 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:04.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:04.927 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.63 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942441 s, 111 MB/s 2024-04-26T20:23:04.994 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:04.994 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:05.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:05.051 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.64 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074637 s, 140 MB/s 2024-04-26T20:23:05.115 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:05.115 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:05.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:05.171 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.65 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:05.234 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:05.235 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:05.235 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838844 s, 125 MB/s 2024-04-26T20:23:05.236 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:05.236 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:05.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:05.292 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.66 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:05.355 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:05.355 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844455 s, 124 MB/s 2024-04-26T20:23:05.356 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:05.356 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:05.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:05.413 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.67 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779307 s, 135 MB/s 2024-04-26T20:23:05.477 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:05.477 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:05.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:05.533 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.68 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:05.595 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:05.595 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:05.595 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660618 s, 159 MB/s 2024-04-26T20:23:05.596 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:05.596 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:05.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:05.652 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.69 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:05.715 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:05.715 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:05.715 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783109 s, 134 MB/s 2024-04-26T20:23:05.716 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:05.716 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:05.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:05.773 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.70 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:05.836 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:05.836 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:05.836 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714375 s, 147 MB/s 2024-04-26T20:23:05.837 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:05.837 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:05.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:05.894 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.71 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067653 s, 155 MB/s 2024-04-26T20:23:05.957 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:05.957 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:06.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:06.013 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.72 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:06.077 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:06.077 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:06.077 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817549 s, 128 MB/s 2024-04-26T20:23:06.077 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:06.078 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:06.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:06.134 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.73 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:06.197 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:06.197 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:06.197 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817925 s, 128 MB/s 2024-04-26T20:23:06.198 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:06.198 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:06.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:06.255 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.74 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709555 s, 148 MB/s 2024-04-26T20:23:06.318 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:06.318 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:06.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:06.374 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.75 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:06.437 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:06.437 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:06.437 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801984 s, 131 MB/s 2024-04-26T20:23:06.439 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:06.439 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:06.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:06.495 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.76 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841422 s, 125 MB/s 2024-04-26T20:23:06.561 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:06.561 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:06.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:06.616 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.77 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:06.681 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846243 s, 124 MB/s 2024-04-26T20:23:06.682 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:06.682 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:06.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:06.739 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.78 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:06.805 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:06.805 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:06.805 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102601 s, 102 MB/s 2024-04-26T20:23:06.806 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:06.806 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:06.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:06.863 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.79 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:06.926 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:06.926 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:06.926 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790105 s, 133 MB/s 2024-04-26T20:23:06.927 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:06.927 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:06.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:06.982 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.80 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:07.048 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949085 s, 110 MB/s 2024-04-26T20:23:07.049 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:07.049 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:07.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:07.104 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.81 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:07.169 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:07.169 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:07.169 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786671 s, 133 MB/s 2024-04-26T20:23:07.169 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:07.169 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:07.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:07.225 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.82 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:07.288 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:07.289 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:07.289 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789123 s, 133 MB/s 2024-04-26T20:23:07.289 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:07.289 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:07.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:07.345 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.83 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:07.411 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:07.412 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:07.412 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107398 s, 97.6 MB/s 2024-04-26T20:23:07.412 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:07.412 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:07.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:07.467 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.84 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799587 s, 131 MB/s 2024-04-26T20:23:07.532 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:07.532 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:07.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:07.588 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.85 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:07.653 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:07.654 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:07.654 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927335 s, 113 MB/s 2024-04-26T20:23:07.655 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:07.655 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:07.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:07.711 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.86 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:07.774 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:07.774 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:07.774 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741592 s, 141 MB/s 2024-04-26T20:23:07.775 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:07.775 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:07.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:07.831 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.87 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:07.896 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:07.896 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:07.896 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00935167 s, 112 MB/s 2024-04-26T20:23:07.897 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:07.897 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:07.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:07.953 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.88 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:08.016 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:08.016 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:08.016 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820036 s, 128 MB/s 2024-04-26T20:23:08.017 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:08.017 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:08.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:08.074 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.89 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745234 s, 141 MB/s 2024-04-26T20:23:08.138 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:08.138 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:08.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:08.194 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.90 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795073 s, 132 MB/s 2024-04-26T20:23:08.258 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:08.258 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:08.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:08.313 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.91 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:08.377 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:08.377 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:08.377 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747331 s, 140 MB/s 2024-04-26T20:23:08.377 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:08.377 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:08.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:08.432 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.92 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:08.499 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103054 s, 102 MB/s 2024-04-26T20:23:08.500 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:08.500 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:08.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:08.556 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.93 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:08.620 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:08.620 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:08.620 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771465 s, 136 MB/s 2024-04-26T20:23:08.620 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:08.620 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:08.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:08.676 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.94 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:08.740 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:08.740 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:08.740 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785958 s, 133 MB/s 2024-04-26T20:23:08.741 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:08.741 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:08.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:08.797 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.95 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:08.862 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:08.862 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:08.862 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773472 s, 136 MB/s 2024-04-26T20:23:08.862 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:08.862 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:08.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:08.918 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.96 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:08.981 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:08.981 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:08.981 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800984 s, 131 MB/s 2024-04-26T20:23:08.982 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:08.982 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:09.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:09.038 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.97 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:09.101 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:09.101 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:09.101 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759139 s, 138 MB/s 2024-04-26T20:23:09.102 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:09.102 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:09.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:09.159 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.98 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:09.221 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:09.221 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:09.221 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722629 s, 145 MB/s 2024-04-26T20:23:09.222 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:09.223 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:09.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:09.278 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.99 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:09.342 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:09.342 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:09.342 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793059 s, 132 MB/s 2024-04-26T20:23:09.343 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:09.343 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:09.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:09.399 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.100 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:09.463 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:09.463 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:09.463 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761445 s, 138 MB/s 2024-04-26T20:23:09.464 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:09.464 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:09.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:09.521 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.101 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:09.585 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:09.585 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:09.586 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810728 s, 129 MB/s 2024-04-26T20:23:09.586 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:09.586 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:09.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:09.642 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.102 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:09.707 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:09.707 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:09.707 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853055 s, 123 MB/s 2024-04-26T20:23:09.708 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:09.708 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:09.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:09.763 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.103 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745482 s, 141 MB/s 2024-04-26T20:23:09.829 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:09.829 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:09.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:09.885 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.104 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:09.948 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:09.949 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:09.949 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762571 s, 138 MB/s 2024-04-26T20:23:09.949 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:09.949 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:10.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:10.005 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.105 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:10.071 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:10.072 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:10.072 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107316 s, 97.7 MB/s 2024-04-26T20:23:10.072 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:10.072 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:10.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:10.128 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.106 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:10.192 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:10.193 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:10.193 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832743 s, 126 MB/s 2024-04-26T20:23:10.193 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:10.193 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:10.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:10.249 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.107 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:10.314 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:10.314 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:10.314 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933781 s, 112 MB/s 2024-04-26T20:23:10.315 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:10.315 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:10.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:10.371 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.108 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:10.432 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641598 s, 163 MB/s 2024-04-26T20:23:10.434 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:10.434 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:10.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:10.490 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.109 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:10.556 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:10.556 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:10.556 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927974 s, 113 MB/s 2024-04-26T20:23:10.557 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:10.557 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:10.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:10.613 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.110 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103462 s, 101 MB/s 2024-04-26T20:23:10.679 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:10.680 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:10.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:10.736 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.111 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:10.799 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:10.799 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:10.799 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727912 s, 144 MB/s 2024-04-26T20:23:10.800 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:10.800 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:10.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:10.856 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.112 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:10.919 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:10.919 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:10.919 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663797 s, 158 MB/s 2024-04-26T20:23:10.919 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:10.920 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:10.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:10.975 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.113 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:11.040 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:11.040 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768445 s, 136 MB/s 2024-04-26T20:23:11.040 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:11.040 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:11.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:11.096 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.114 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:11.159 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:11.159 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:11.159 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725262 s, 145 MB/s 2024-04-26T20:23:11.159 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:11.159 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:11.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:11.216 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.115 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:11.279 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:11.279 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:11.279 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789356 s, 133 MB/s 2024-04-26T20:23:11.280 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:11.281 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:11.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:11.337 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.116 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101432 s, 103 MB/s 2024-04-26T20:23:11.403 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:11.404 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:11.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:11.460 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.117 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723488 s, 145 MB/s 2024-04-26T20:23:11.523 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:11.524 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:11.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:11.580 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.118 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:11.643 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:11.643 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:11.643 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074705 s, 140 MB/s 2024-04-26T20:23:11.643 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:11.644 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:11.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:11.700 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.119 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:11.764 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:11.764 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:11.764 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772835 s, 136 MB/s 2024-04-26T20:23:11.764 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:11.764 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:11.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:11.819 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.120 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:11.882 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:11.882 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:11.883 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006228 s, 168 MB/s 2024-04-26T20:23:11.883 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:11.883 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:11.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:11.939 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.121 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831331 s, 126 MB/s 2024-04-26T20:23:12.003 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:12.004 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:12.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:12.060 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.122 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103445 s, 101 MB/s 2024-04-26T20:23:12.126 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:12.126 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:12.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:12.181 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.123 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:12.246 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:12.246 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:12.246 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843042 s, 124 MB/s 2024-04-26T20:23:12.246 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:12.246 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:12.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:12.303 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.124 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:12.366 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:12.366 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:12.366 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720095 s, 146 MB/s 2024-04-26T20:23:12.366 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:12.366 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:12.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:12.423 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.125 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078433 s, 134 MB/s 2024-04-26T20:23:12.487 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:12.487 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:12.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:12.543 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.126 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:12.607 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:12.607 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:12.607 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802126 s, 131 MB/s 2024-04-26T20:23:12.607 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:12.607 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:12.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:12.663 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.127 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:12.727 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:12.727 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:12.727 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803316 s, 131 MB/s 2024-04-26T20:23:12.727 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:12.727 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:12.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:12.783 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.128 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:12.849 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:12.849 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:12.849 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104018 s, 101 MB/s 2024-04-26T20:23:12.849 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:12.850 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:12.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:12.906 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.129 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:12.968 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:12.969 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:12.969 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743969 s, 141 MB/s 2024-04-26T20:23:12.970 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:12.970 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:13.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:13.026 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.130 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:13.090 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:13.090 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:13.090 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853275 s, 123 MB/s 2024-04-26T20:23:13.091 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:13.091 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:13.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:13.147 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.131 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853966 s, 123 MB/s 2024-04-26T20:23:13.213 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:13.213 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:13.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:13.269 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.132 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745576 s, 141 MB/s 2024-04-26T20:23:13.333 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:13.333 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:13.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:13.389 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.133 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:13.452 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:13.452 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:13.452 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724403 s, 145 MB/s 2024-04-26T20:23:13.452 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:13.452 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:13.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:13.507 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.134 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:13.574 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:13.574 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:13.574 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960535 s, 109 MB/s 2024-04-26T20:23:13.574 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:13.574 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:13.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:13.629 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.135 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:13.697 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:13.697 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:13.697 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112375 s, 93.3 MB/s 2024-04-26T20:23:13.698 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:13.698 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:13.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:13.754 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.136 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698038 s, 150 MB/s 2024-04-26T20:23:13.817 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:13.817 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:13.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:13.873 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.137 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783771 s, 134 MB/s 2024-04-26T20:23:13.937 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:13.937 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:13.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:13.992 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.138 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:14.055 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790352 s, 133 MB/s 2024-04-26T20:23:14.056 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:14.056 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:14.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:14.112 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.139 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:14.176 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:14.176 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:14.176 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807819 s, 130 MB/s 2024-04-26T20:23:14.176 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:14.177 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:14.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:14.233 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.140 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:14.296 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:14.297 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:14.297 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784256 s, 134 MB/s 2024-04-26T20:23:14.297 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:14.297 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:14.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:14.353 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.141 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:14.416 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:14.416 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:14.416 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739407 s, 142 MB/s 2024-04-26T20:23:14.417 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:14.417 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:14.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:14.473 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.142 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724159 s, 145 MB/s 2024-04-26T20:23:14.538 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:14.538 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:14.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:14.593 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.143 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:14.657 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:14.657 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:14.657 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083612 s, 125 MB/s 2024-04-26T20:23:14.658 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:14.658 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:14.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:14.715 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.144 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:14.780 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:14.780 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:14.780 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632634 s, 166 MB/s 2024-04-26T20:23:14.781 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:14.781 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:14.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:14.837 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.145 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:14.900 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:14.900 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:14.900 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806204 s, 130 MB/s 2024-04-26T20:23:14.901 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:14.901 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:14.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:14.958 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.146 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:15.023 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067718 s, 155 MB/s 2024-04-26T20:23:15.090 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:15.091 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:15.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:15.106 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.147 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078343 s, 134 MB/s 2024-04-26T20:23:15.171 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:15.171 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:15.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:15.226 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.148 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815921 s, 129 MB/s 2024-04-26T20:23:15.291 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:15.292 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:15.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:15.347 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.149 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:15.410 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:15.410 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:15.410 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667692 s, 157 MB/s 2024-04-26T20:23:15.411 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:15.411 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:15.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:15.468 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.150 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:15.530 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:15.530 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:15.530 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648468 s, 162 MB/s 2024-04-26T20:23:15.531 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:15.531 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:15.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:15.587 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.151 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:15.651 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:15.651 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:15.651 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853219 s, 123 MB/s 2024-04-26T20:23:15.652 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:15.652 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:15.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:15.708 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.152 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:15.772 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:15.772 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:15.772 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856151 s, 122 MB/s 2024-04-26T20:23:15.773 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:15.773 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:15.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:15.829 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.153 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:15.893 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:15.893 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:15.893 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821186 s, 128 MB/s 2024-04-26T20:23:15.894 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:15.894 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:15.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:15.951 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.154 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:16.015 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:16.015 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:16.015 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773765 s, 136 MB/s 2024-04-26T20:23:16.015 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:16.015 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:16.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:16.071 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.155 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:16.133 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:16.133 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:16.133 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732856 s, 143 MB/s 2024-04-26T20:23:16.134 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:16.134 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:16.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:16.191 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.156 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:16.254 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:16.255 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:16.255 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772864 s, 136 MB/s 2024-04-26T20:23:16.255 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:16.255 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:16.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:16.310 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.157 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:16.375 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:16.375 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826183 s, 127 MB/s 2024-04-26T20:23:16.376 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:16.376 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:16.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:16.432 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.158 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:16.495 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:16.495 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:16.495 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702746 s, 149 MB/s 2024-04-26T20:23:16.495 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:16.495 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:16.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:16.551 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.159 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:16.616 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801513 s, 131 MB/s 2024-04-26T20:23:16.616 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:16.616 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:16.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:16.673 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.160 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:16.739 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:16.739 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:16.739 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100224 s, 105 MB/s 2024-04-26T20:23:16.740 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:16.740 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:16.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:16.796 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.161 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:16.861 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:16.861 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:16.861 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856086 s, 122 MB/s 2024-04-26T20:23:16.862 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:16.862 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:16.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:16.917 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.162 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781737 s, 134 MB/s 2024-04-26T20:23:16.982 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:16.982 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:17.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:17.038 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.163 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:17.102 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:17.102 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:17.102 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757389 s, 138 MB/s 2024-04-26T20:23:17.103 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:17.103 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:17.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:17.159 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.164 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:17.221 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:17.221 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721847 s, 145 MB/s 2024-04-26T20:23:17.223 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:17.223 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:17.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:17.279 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.165 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:17.343 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:17.343 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781188 s, 134 MB/s 2024-04-26T20:23:17.343 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:17.343 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:17.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:17.398 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.166 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772359 s, 136 MB/s 2024-04-26T20:23:17.463 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:17.463 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:17.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:17.519 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.167 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:17.581 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:17.581 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:17.581 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641645 s, 163 MB/s 2024-04-26T20:23:17.582 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:17.583 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:17.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:17.639 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.168 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:17.703 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895539 s, 117 MB/s 2024-04-26T20:23:17.707 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:17.707 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:17.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:17.764 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.169 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:17.828 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:17.829 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:17.829 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807711 s, 130 MB/s 2024-04-26T20:23:17.829 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:17.829 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:17.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:17.885 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.170 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006165 s, 170 MB/s 2024-04-26T20:23:17.948 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:17.948 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:18.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:18.006 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.171 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825167 s, 127 MB/s 2024-04-26T20:23:18.071 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:18.071 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:18.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:18.130 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.172 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:18.194 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:18.194 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:18.194 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840137 s, 125 MB/s 2024-04-26T20:23:18.195 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:18.195 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:18.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:18.251 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.173 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:18.314 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:18.314 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:18.314 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733889 s, 143 MB/s 2024-04-26T20:23:18.318 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:18.318 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:18.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:18.370 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.174 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:18.436 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:18.437 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:18.437 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881159 s, 119 MB/s 2024-04-26T20:23:18.437 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:18.437 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:18.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:18.493 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.175 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685847 s, 153 MB/s 2024-04-26T20:23:18.557 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:18.557 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:18.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:18.613 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.176 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:18.678 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:18.679 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:18.679 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905858 s, 116 MB/s 2024-04-26T20:23:18.680 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:18.680 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:18.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:18.736 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.177 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718566 s, 146 MB/s 2024-04-26T20:23:18.800 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:18.800 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:18.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:18.857 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.178 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:18.923 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:18.923 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:18.923 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819454 s, 128 MB/s 2024-04-26T20:23:18.924 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:18.924 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:18.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:18.979 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.179 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:19.044 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:19.044 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:19.044 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839003 s, 125 MB/s 2024-04-26T20:23:19.044 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:19.044 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:19.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:19.100 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.180 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:19.163 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:19.163 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:19.164 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748027 s, 140 MB/s 2024-04-26T20:23:19.164 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:19.164 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:19.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:19.220 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.181 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:19.286 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:19.286 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:19.286 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994122 s, 105 MB/s 2024-04-26T20:23:19.286 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:19.286 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:19.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:19.344 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.182 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796005 s, 132 MB/s 2024-04-26T20:23:19.408 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:19.408 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:19.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:19.464 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.183 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:19.514 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:23:19.528 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:19.529 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:19.529 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842767 s, 124 MB/s 2024-04-26T20:23:19.529 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:19.529 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:19.530 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:23:19.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:19.625 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.184 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:19.690 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:19.690 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:19.690 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820581 s, 128 MB/s 2024-04-26T20:23:19.691 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:19.691 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:19.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:19.751 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.185 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799151 s, 131 MB/s 2024-04-26T20:23:19.815 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:19.816 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:19.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:19.872 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.186 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:19.936 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:19.936 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:19.936 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822651 s, 127 MB/s 2024-04-26T20:23:19.937 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:19.937 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:19.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:19.992 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.187 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:20.057 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:20.058 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:20.058 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091038 s, 115 MB/s 2024-04-26T20:23:20.059 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:20.059 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:20.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:20.115 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.188 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:20.179 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:20.179 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:20.179 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805585 s, 130 MB/s 2024-04-26T20:23:20.179 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:20.179 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:20.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:20.234 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.189 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:20.303 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117645 s, 89.1 MB/s 2024-04-26T20:23:20.303 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:20.303 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:20.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:20.359 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.190 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:20.423 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:20.423 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:20.423 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777717 s, 135 MB/s 2024-04-26T20:23:20.423 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:20.423 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:20.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:20.479 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.191 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:20.543 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:20.543 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:20.543 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777425 s, 135 MB/s 2024-04-26T20:23:20.543 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:20.543 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:20.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:20.599 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.192 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:20.663 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:20.663 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:20.663 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882409 s, 119 MB/s 2024-04-26T20:23:20.664 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:20.664 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:20.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:20.720 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.193 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:20.783 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:20.784 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:20.784 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737921 s, 142 MB/s 2024-04-26T20:23:20.784 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:20.784 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:20.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:20.840 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.194 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:20.904 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:20.904 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:20.904 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796361 s, 132 MB/s 2024-04-26T20:23:20.904 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:20.904 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:20.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:20.961 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.195 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:21.025 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:21.025 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780218 s, 134 MB/s 2024-04-26T20:23:21.026 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:21.026 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:21.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:21.083 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.196 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:21.146 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:21.146 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:21.146 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815142 s, 129 MB/s 2024-04-26T20:23:21.147 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:21.147 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:21.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:21.203 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.197 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:21.266 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:21.266 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:21.266 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700977 s, 150 MB/s 2024-04-26T20:23:21.267 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:21.267 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:21.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:21.323 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.198 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824164 s, 127 MB/s 2024-04-26T20:23:21.392 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:21.392 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:21.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:21.448 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol19350/999f3f42-0a27-4f3c-ac56-41987a234e85/subvolume_file.199 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T20:23:21.513 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T20:23:21.513 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T20:23:21.514 INFO:teuthology.orchestra.run.smithi037.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788736 s, 133 MB/s 2024-04-26T20:23:21.514 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol19350 subvol_snap85010 2024-04-26T20:23:21.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.667+0000 7f883455f640 1 Processor -- start 2024-04-26T20:23:21.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.668+0000 7f883455f640 1 -- start start 2024-04-26T20:23:21.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.668+0000 7f883455f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f882c0ce270 0x7f882c15ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:21.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.668+0000 7f883455f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc4a0 0x7f882c0c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:21.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.668+0000 7f883455f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f882c0ced10 con 0x7f882c0cc920 2024-04-26T20:23:21.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.668+0000 7f883455f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f882c0cf920 con 0x7f882c0ce270 2024-04-26T20:23:21.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.668+0000 7f883455f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f882c0ceff0 con 0x7f882c0cc4a0 2024-04-26T20:23:21.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f8831afa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc4a0 0x7f882c0c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:21.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f88322fb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f882c0ce270 0x7f882c15ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:21.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f8832afc640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f882c0cc920 0x7f882c0cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:53774/0 (socket says 172.21.15.37:53774) 2024-04-26T20:23:21.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f8831afa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc4a0 0x7f882c0c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:38892/0 (socket says 172.21.15.37:38892) 2024-04-26T20:23:21.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f8832afc640 1 -- 172.21.15.37:0/643870492 learned_addr learned my addr 172.21.15.37:0/643870492 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:21.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f88312f9640 1 -- 172.21.15.37:0/643870492 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2687005893 0 0) 0x7f882c0ced10 con 0x7f882c0cc920 2024-04-26T20:23:21.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f88312f9640 1 -- 172.21.15.37:0/643870492 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8818005730 con 0x7f882c0cc920 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f88322fb640 1 -- 172.21.15.37:0/643870492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc4a0 msgr2=0x7f882c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f88322fb640 1 --2- 172.21.15.37:0/643870492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc4a0 0x7f882c0c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f88322fb640 1 -- 172.21.15.37:0/643870492 >> v1:172.21.15.134:6789/0 conn(0x7f882c0cc920 legacy=0x7f882c0cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f88322fb640 1 -- 172.21.15.37:0/643870492 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882c0cf2d0 con 0x7f882c0ce270 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.669+0000 7f8831afa640 1 --2- 172.21.15.37:0/643870492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc4a0 0x7f882c0c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.670+0000 7f88322fb640 1 --2- 172.21.15.37:0/643870492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f882c0ce270 0x7f882c15ce60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f88280097a0 tx=0x7f882802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6918fe8dab7db173 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.670+0000 7f88312f9640 1 -- 172.21.15.37:0/643870492 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828037070 con 0x7f882c0ce270 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.670+0000 7f88312f9640 1 -- 172.21.15.37:0/643870492 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8828032440 con 0x7f882c0ce270 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.670+0000 7f88312f9640 1 -- 172.21.15.37:0/643870492 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88280327f0 con 0x7f882c0ce270 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.671+0000 7f883455f640 1 -- 172.21.15.37:0/643870492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f882c0ce270 msgr2=0x7f882c15ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.671+0000 7f883455f640 1 --2- 172.21.15.37:0/643870492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f882c0ce270 0x7f882c15ce60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f88280097a0 tx=0x7f882802a180 comp rx=0 tx=0).stop 2024-04-26T20:23:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.671+0000 7f883455f640 1 -- 172.21.15.37:0/643870492 shutdown_connections 2024-04-26T20:23:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.671+0000 7f883455f640 1 --2- 172.21.15.37:0/643870492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc4a0 0x7f882c0c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.671+0000 7f883455f640 1 --2- 172.21.15.37:0/643870492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f882c0ce270 0x7f882c15ce60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.671+0000 7f883455f640 1 -- 172.21.15.37:0/643870492 >> 172.21.15.37:0/643870492 conn(0x7f882c0b2c60 msgr2=0x7f882c0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.671+0000 7f883455f640 1 -- 172.21.15.37:0/643870492 shutdown_connections 2024-04-26T20:23:21.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.671+0000 7f883455f640 1 -- 172.21.15.37:0/643870492 wait complete. 2024-04-26T20:23:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.672+0000 7f883455f640 1 Processor -- start 2024-04-26T20:23:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.672+0000 7f883455f640 1 -- start start 2024-04-26T20:23:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f883455f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f882c0cc4a0 0x7f882c0c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f883455f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc920 0x7f882c15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f883455f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c0ce270 0x7f882c14b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f883455f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f882c0ce9e0 con 0x7f882c0ce270 2024-04-26T20:23:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f883455f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f882c0ce650 con 0x7f882c0cc4a0 2024-04-26T20:23:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f883455f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f882c06f3a0 con 0x7f882c0cc920 2024-04-26T20:23:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f8832afc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c0ce270 0x7f882c14b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f8832afc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c0ce270 0x7f882c14b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51572/0 (socket says 172.21.15.37:51572) 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f8832afc640 1 -- 172.21.15.37:0/2349096131 learned_addr learned my addr 172.21.15.37:0/2349096131 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f88322fb640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f882c0cc4a0 0x7f882c0c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f8831afa640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc920 0x7f882c15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f8832afc640 1 -- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc920 msgr2=0x7f882c15dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f8832afc640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc920 0x7f882c15dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f8832afc640 1 -- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f882c0cc4a0 msgr2=0x7f882c0c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f8832afc640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f882c0cc4a0 0x7f882c0c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.673+0000 7f8832afc640 1 -- 172.21.15.37:0/2349096131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882c15aba0 con 0x7f882c0ce270 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.674+0000 7f8832afc640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c0ce270 0x7f882c14b3d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f882400caa0 tx=0x7f882400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.674+0000 7f8822ffd640 1 -- 172.21.15.37:0/2349096131 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824017070 con 0x7f882c0ce270 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.674+0000 7f8822ffd640 1 -- 172.21.15.37:0/2349096131 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882400ac00 con 0x7f882c0ce270 2024-04-26T20:23:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.674+0000 7f883455f640 1 -- 172.21.15.37:0/2349096131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882c15ae80 con 0x7f882c0ce270 2024-04-26T20:23:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.674+0000 7f8822ffd640 1 -- 172.21.15.37:0/2349096131 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824012400 con 0x7f882c0ce270 2024-04-26T20:23:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.674+0000 7f883455f640 1 -- 172.21.15.37:0/2349096131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f882c15b440 con 0x7f882c0ce270 2024-04-26T20:23:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.675+0000 7f883455f640 1 -- 172.21.15.37:0/2349096131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f882c069770 con 0x7f882c0ce270 2024-04-26T20:23:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.675+0000 7f8822ffd640 1 -- 172.21.15.37:0/2349096131 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f882401a070 con 0x7f882c0ce270 2024-04-26T20:23:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.676+0000 7f8822ffd640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f881006c980 0x7f881006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.676+0000 7f8822ffd640 1 -- 172.21.15.37:0/2349096131 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f882408b870 con 0x7f882c0ce270 2024-04-26T20:23:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.676+0000 7f88322fb640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f881006c980 0x7f881006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.677+0000 7f88322fb640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f881006c980 0x7f881006ee40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8828034000 tx=0x7f882802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:21.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.680+0000 7f8822ffd640 1 -- 172.21.15.37:0/2349096131 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f882c069770 con 0x7f882c0ce270 2024-04-26T20:23:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.899+0000 7f883455f640 1 -- 172.21.15.37:0/2349096131 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol19350", "snap_name": "subvol_snap85010", "target": ["mon-mgr", ""]}) -- 0x7f882c06a360 con 0x7f881006c980 2024-04-26T20:23:21.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.916+0000 7f8822ffd640 1 -- 172.21.15.37:0/2349096131 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f882c06a360 con 0x7f881006c980 2024-04-26T20:23:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 -- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f881006c980 msgr2=0x7f881006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f881006c980 0x7f881006ee40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8828034000 tx=0x7f882802e000 comp rx=0 tx=0).stop 2024-04-26T20:23:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 -- 172.21.15.37:0/2349096131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c0ce270 msgr2=0x7f882c14b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c0ce270 0x7f882c14b3d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f882400caa0 tx=0x7f882400cf60 comp rx=0 tx=0).stop 2024-04-26T20:23:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 -- 172.21.15.37:0/2349096131 shutdown_connections 2024-04-26T20:23:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c0ce270 0x7f882c14b3d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f881006c980 0x7f881006ee40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f882c0cc920 0x7f882c15dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 --2- 172.21.15.37:0/2349096131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f882c0cc4a0 0x7f882c0c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 -- 172.21.15.37:0/2349096131 >> 172.21.15.37:0/2349096131 conn(0x7f882c0b2c60 msgr2=0x7f882c0b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 -- 172.21.15.37:0/2349096131 shutdown_connections 2024-04-26T20:23:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:21.919+0000 7f883455f640 1 -- 172.21.15.37:0/2349096131 wait complete. 2024-04-26T20:23:21.935 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_delay 5 2024-04-26T20:23:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.039+0000 7f004df2b640 1 Processor -- start 2024-04-26T20:23:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.040+0000 7f004df2b640 1 -- start start 2024-04-26T20:23:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.040+0000 7f004df2b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0048160800 0x7f00481610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.040+0000 7f004df2b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d5b60 0x7f00480d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.040+0000 7f004df2b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00480d0930 con 0x7f00480d2150 2024-04-26T20:23:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.040+0000 7f004df2b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f00480d2590 con 0x7f0048160800 2024-04-26T20:23:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.040+0000 7f004df2b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f00480d0c00 con 0x7f00480d5b60 2024-04-26T20:23:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f0046ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d5b60 0x7f00480d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f00477fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0048160800 0x7f00481610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f0046ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d5b60 0x7f00480d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:38914/0 (socket says 172.21.15.37:38914) 2024-04-26T20:23:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f0046ffd640 1 -- 172.21.15.37:0/3723830855 learned_addr learned my addr 172.21.15.37:0/3723830855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f0047fff640 1 --1- 172.21.15.37:0/3723830855 >> v1:172.21.15.134:6789/0 conn(0x7f00480d2150 0x7f00480d3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:53788/0 (socket says 172.21.15.37:53788) 2024-04-26T20:23:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f00467fc640 1 -- 172.21.15.37:0/3723830855 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3015424620 0 0) 0x7f00480d0930 con 0x7f00480d2150 2024-04-26T20:23:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f00467fc640 1 -- 172.21.15.37:0/3723830855 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0030003520 con 0x7f00480d2150 2024-04-26T20:23:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f0046ffd640 1 -- 172.21.15.37:0/3723830855 >> v2:172.21.15.134:3300/0 conn(0x7f0048160800 msgr2=0x7f00481610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f0046ffd640 1 --2- 172.21.15.37:0/3723830855 >> v2:172.21.15.134:3300/0 conn(0x7f0048160800 0x7f00481610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f0046ffd640 1 -- 172.21.15.37:0/3723830855 >> v1:172.21.15.134:6789/0 conn(0x7f00480d2150 legacy=0x7f00480d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f0046ffd640 1 -- 172.21.15.37:0/3723830855 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00480d0ee0 con 0x7f00480d5b60 2024-04-26T20:23:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.041+0000 7f00477fe640 1 --2- 172.21.15.37:0/3723830855 >> v2:172.21.15.134:3300/0 conn(0x7f0048160800 0x7f00481610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.042+0000 7f0046ffd640 1 --2- 172.21.15.37:0/3723830855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d5b60 0x7f00480d1c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f003c009870 tx=0x7f003c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db113c8da4fb9c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.042+0000 7f00467fc640 1 -- 172.21.15.37:0/3723830855 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c037070 con 0x7f00480d5b60 2024-04-26T20:23:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.042+0000 7f00467fc640 1 -- 172.21.15.37:0/3723830855 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003c032440 con 0x7f00480d5b60 2024-04-26T20:23:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.042+0000 7f00467fc640 1 -- 172.21.15.37:0/3723830855 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c0327f0 con 0x7f00480d5b60 2024-04-26T20:23:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.043+0000 7f004df2b640 1 -- 172.21.15.37:0/3723830855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d5b60 msgr2=0x7f00480d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.043+0000 7f004df2b640 1 --2- 172.21.15.37:0/3723830855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d5b60 0x7f00480d1c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f003c009870 tx=0x7f003c02a270 comp rx=0 tx=0).stop 2024-04-26T20:23:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.043+0000 7f004df2b640 1 -- 172.21.15.37:0/3723830855 shutdown_connections 2024-04-26T20:23:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.043+0000 7f004df2b640 1 --2- 172.21.15.37:0/3723830855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d5b60 0x7f00480d1c10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.043+0000 7f004df2b640 1 --2- 172.21.15.37:0/3723830855 >> v2:172.21.15.134:3300/0 conn(0x7f0048160800 0x7f00481610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.043+0000 7f004df2b640 1 -- 172.21.15.37:0/3723830855 >> 172.21.15.37:0/3723830855 conn(0x7f004809e550 msgr2=0x7f00480d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.043+0000 7f004df2b640 1 -- 172.21.15.37:0/3723830855 shutdown_connections 2024-04-26T20:23:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.043+0000 7f004df2b640 1 -- 172.21.15.37:0/3723830855 wait complete. 2024-04-26T20:23:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.044+0000 7f004df2b640 1 Processor -- start 2024-04-26T20:23:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.044+0000 7f004df2b640 1 -- start start 2024-04-26T20:23:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f004df2b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d2150 0x7f00480b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f004df2b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00480d5b60 0x7f0048161ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f004df2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0048160800 0x7f00481621e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f004df2b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00480d30f0 con 0x7f0048160800 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f004df2b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00480d2e10 con 0x7f00480d5b60 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f004df2b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f00480b6300 con 0x7f00480d2150 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f0047fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0048160800 0x7f00481621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f0047fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0048160800 0x7f00481621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51576/0 (socket says 172.21.15.37:51576) 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f0047fff640 1 -- 172.21.15.37:0/3656829225 learned_addr learned my addr 172.21.15.37:0/3656829225 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f0046ffd640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00480d5b60 0x7f0048161ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f00477fe640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d2150 0x7f00480b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f0047fff640 1 -- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d2150 msgr2=0x7f00480b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f0047fff640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d2150 0x7f00480b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f0047fff640 1 -- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00480d5b60 msgr2=0x7f0048161ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f0047fff640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00480d5b60 0x7f0048161ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f0047fff640 1 -- 172.21.15.37:0/3656829225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00481f8ff0 con 0x7f0048160800 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.045+0000 7f0047fff640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0048160800 0x7f00481621e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f004000c700 tx=0x7f004000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.046+0000 7f002ffff640 1 -- 172.21.15.37:0/3656829225 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040017070 con 0x7f0048160800 2024-04-26T20:23:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.046+0000 7f002ffff640 1 -- 172.21.15.37:0/3656829225 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f004000ce10 con 0x7f0048160800 2024-04-26T20:23:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.046+0000 7f004df2b640 1 -- 172.21.15.37:0/3656829225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00480b9cf0 con 0x7f0048160800 2024-04-26T20:23:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.046+0000 7f002ffff640 1 -- 172.21.15.37:0/3656829225 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004000a860 con 0x7f0048160800 2024-04-26T20:23:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.046+0000 7f004df2b640 1 -- 172.21.15.37:0/3656829225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00481f9570 con 0x7f0048160800 2024-04-26T20:23:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.047+0000 7f004df2b640 1 -- 172.21.15.37:0/3656829225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004806a360 con 0x7f0048160800 2024-04-26T20:23:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.047+0000 7f002ffff640 1 -- 172.21.15.37:0/3656829225 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f004001a070 con 0x7f0048160800 2024-04-26T20:23:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.048+0000 7f002ffff640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f003406c980 0x7f003406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.048+0000 7f002ffff640 1 -- 172.21.15.37:0/3656829225 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f004008b170 con 0x7f0048160800 2024-04-26T20:23:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.048+0000 7f00477fe640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f003406c980 0x7f003406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.048+0000 7f00477fe640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f003406c980 0x7f003406ee40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0038009560 tx=0x7f0038009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:22.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.052+0000 7f002ffff640 1 -- 172.21.15.37:0/3656829225 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0040058f60 con 0x7f0048160800 2024-04-26T20:23:22.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.236+0000 7f004df2b640 1 -- 172.21.15.37:0/3656829225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}] v 0) -- 0x7f00480d4530 con 0x7f0048160800 2024-04-26T20:23:22.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.239+0000 7f002ffff640 1 -- 172.21.15.37:0/3656829225 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}]=0 v11)=0 v11) ==== 130+0+0 (secure 0 0 0) 0x7f004005d260 con 0x7f0048160800 2024-04-26T20:23:22.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.240+0000 7f004df2b640 1 -- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f003406c980 msgr2=0x7f003406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.240+0000 7f004df2b640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f003406c980 0x7f003406ee40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0038009560 tx=0x7f0038009190 comp rx=0 tx=0).stop 2024-04-26T20:23:22.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.241+0000 7f004df2b640 1 -- 172.21.15.37:0/3656829225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0048160800 msgr2=0x7f00481621e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.241+0000 7f004df2b640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0048160800 0x7f00481621e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f004000c700 tx=0x7f004000cbc0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.241+0000 7f004df2b640 1 -- 172.21.15.37:0/3656829225 shutdown_connections 2024-04-26T20:23:22.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.241+0000 7f004df2b640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0048160800 0x7f00481621e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.241+0000 7f004df2b640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00480d5b60 0x7f0048161ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.241+0000 7f004df2b640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f003406c980 0x7f003406ee40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.241+0000 7f004df2b640 1 --2- 172.21.15.37:0/3656829225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00480d2150 0x7f00480b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.241+0000 7f004df2b640 1 -- 172.21.15.37:0/3656829225 >> 172.21.15.37:0/3656829225 conn(0x7f004809e550 msgr2=0x7f004815d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:22.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.241+0000 7f004df2b640 1 -- 172.21.15.37:0/3656829225 shutdown_connections 2024-04-26T20:23:22.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.241+0000 7f004df2b640 1 -- 172.21.15.37:0/3656829225 wait complete. 2024-04-26T20:23:22.251 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol19350 subvol_snap85010 subvol_clone57050 2024-04-26T20:23:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.358+0000 7fe57a19d640 1 Processor -- start 2024-04-26T20:23:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.358+0000 7fe57a19d640 1 -- start start 2024-04-26T20:23:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.358+0000 7fe57a19d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe57415d460 0x7fe57406e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.358+0000 7fe57a19d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe574073c40 0x7fe5740ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.358+0000 7fe57a19d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5740cd100 0x7fe5740cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe57a19d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fe5740add80 con 0x7fe57415d460 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe57a19d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe57406ece0 con 0x7fe574073c40 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe57a19d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe57406efc0 con 0x7fe5740cd100 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe5737fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe57415d460 0x7fe57406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe5737fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe57415d460 0x7fe57406e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51588/0 (socket says 172.21.15.37:51588) 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe5737fe640 1 -- 172.21.15.37:0/953880997 learned_addr learned my addr 172.21.15.37:0/953880997 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe572ffd640 1 --2- 172.21.15.37:0/953880997 >> v2:172.21.15.134:3300/0 conn(0x7fe574073c40 0x7fe5740ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe573fff640 1 --2- 172.21.15.37:0/953880997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5740cd100 0x7fe5740cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe5737fe640 1 -- 172.21.15.37:0/953880997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5740cd100 msgr2=0x7fe5740cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe5737fe640 1 --2- 172.21.15.37:0/953880997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5740cd100 0x7fe5740cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe5737fe640 1 -- 172.21.15.37:0/953880997 >> v2:172.21.15.134:3300/0 conn(0x7fe574073c40 msgr2=0x7fe5740ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe5737fe640 1 --2- 172.21.15.37:0/953880997 >> v2:172.21.15.134:3300/0 conn(0x7fe574073c40 0x7fe5740ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.359+0000 7fe5737fe640 1 -- 172.21.15.37:0/953880997 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57406f2a0 con 0x7fe57415d460 2024-04-26T20:23:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.360+0000 7fe5737fe640 1 --2- 172.21.15.37:0/953880997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe57415d460 0x7fe57406e700 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe574073210 tx=0x7fe56402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5939ca27dd2849af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.360+0000 7fe5727fc640 1 -- 172.21.15.37:0/953880997 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564037070 con 0x7fe57415d460 2024-04-26T20:23:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.360+0000 7fe5727fc640 1 -- 172.21.15.37:0/953880997 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe564032440 con 0x7fe57415d460 2024-04-26T20:23:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.360+0000 7fe5727fc640 1 -- 172.21.15.37:0/953880997 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564032810 con 0x7fe57415d460 2024-04-26T20:23:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.361+0000 7fe57a19d640 1 -- 172.21.15.37:0/953880997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe57415d460 msgr2=0x7fe57406e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.361+0000 7fe57a19d640 1 --2- 172.21.15.37:0/953880997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe57415d460 0x7fe57406e700 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe574073210 tx=0x7fe56402a270 comp rx=0 tx=0).stop 2024-04-26T20:23:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.361+0000 7fe57a19d640 1 -- 172.21.15.37:0/953880997 shutdown_connections 2024-04-26T20:23:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.361+0000 7fe57a19d640 1 --2- 172.21.15.37:0/953880997 >> v2:172.21.15.134:3300/0 conn(0x7fe574073c40 0x7fe5740ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.361+0000 7fe57a19d640 1 --2- 172.21.15.37:0/953880997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5740cd100 0x7fe5740cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.361+0000 7fe57a19d640 1 --2- 172.21.15.37:0/953880997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe57415d460 0x7fe57406e700 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.361+0000 7fe57a19d640 1 -- 172.21.15.37:0/953880997 >> 172.21.15.37:0/953880997 conn(0x7fe5740d0d70 msgr2=0x7fe5740d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.361+0000 7fe57a19d640 1 -- 172.21.15.37:0/953880997 shutdown_connections 2024-04-26T20:23:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.361+0000 7fe57a19d640 1 -- 172.21.15.37:0/953880997 wait complete. 2024-04-26T20:23:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.362+0000 7fe57a19d640 1 Processor -- start 2024-04-26T20:23:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.362+0000 7fe57a19d640 1 -- start start 2024-04-26T20:23:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe57a19d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe574073c40 0x7fe5741f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe57a19d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5740cd100 0x7fe5740cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe57a19d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe57415d460 0x7fe5740d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe57a19d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe57415d9c0 con 0x7fe57415d460 2024-04-26T20:23:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe57a19d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe57415d840 con 0x7fe5740cd100 2024-04-26T20:23:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe57a19d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe5740740a0 con 0x7fe574073c40 2024-04-26T20:23:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe573fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe57415d460 0x7fe5740d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe573fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe57415d460 0x7fe5740d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51600/0 (socket says 172.21.15.37:51600) 2024-04-26T20:23:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe573fff640 1 -- 172.21.15.37:0/1063927863 learned_addr learned my addr 172.21.15.37:0/1063927863 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe572ffd640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5740cd100 0x7fe5740cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe5737fe640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe574073c40 0x7fe5741f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe573fff640 1 -- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe574073c40 msgr2=0x7fe5741f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe573fff640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe574073c40 0x7fe5741f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe573fff640 1 -- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5740cd100 msgr2=0x7fe5740cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe573fff640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5740cd100 0x7fe5740cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe573fff640 1 -- 172.21.15.37:0/1063927863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5740d0820 con 0x7fe57415d460 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.363+0000 7fe573fff640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe57415d460 0x7fe5740d01f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe56c007e40 tx=0x7fe56c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.364+0000 7fe55bfff640 1 -- 172.21.15.37:0/1063927863 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c014620 con 0x7fe57415d460 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.364+0000 7fe55bfff640 1 -- 172.21.15.37:0/1063927863 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56c019070 con 0x7fe57415d460 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.364+0000 7fe57a19d640 1 -- 172.21.15.37:0/1063927863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5741f5170 con 0x7fe57415d460 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.364+0000 7fe55bfff640 1 -- 172.21.15.37:0/1063927863 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c014a20 con 0x7fe57415d460 2024-04-26T20:23:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.364+0000 7fe57a19d640 1 -- 172.21.15.37:0/1063927863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5741f5730 con 0x7fe57415d460 2024-04-26T20:23:22.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.365+0000 7fe57a19d640 1 -- 172.21.15.37:0/1063927863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe57406a360 con 0x7fe57415d460 2024-04-26T20:23:22.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.365+0000 7fe55bfff640 1 -- 172.21.15.37:0/1063927863 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe56c014bc0 con 0x7fe57415d460 2024-04-26T20:23:22.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.366+0000 7fe55bfff640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe56006c980 0x7fe56006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.366+0000 7fe55bfff640 1 -- 172.21.15.37:0/1063927863 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe56c08c480 con 0x7fe57415d460 2024-04-26T20:23:22.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.366+0000 7fe5737fe640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe56006c980 0x7fe56006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.366+0000 7fe5737fe640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe56006c980 0x7fe56006ee40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe564034000 tx=0x7fe56402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:22.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.370+0000 7fe55bfff640 1 -- 172.21.15.37:0/1063927863 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe56c05a840 con 0x7fe57415d460 2024-04-26T20:23:22.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.593+0000 7fe57a19d640 1 -- 172.21.15.37:0/1063927863 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol19350", "snap_name": "subvol_snap85010", "target_sub_name": "subvol_clone57050", "target": ["mon-mgr", ""]}) -- 0x7fe5741f5900 con 0x7fe56006c980 2024-04-26T20:23:22.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.678+0000 7fe55bfff640 1 -- 172.21.15.37:0/1063927863 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe5741f5900 con 0x7fe56006c980 2024-04-26T20:23:22.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 -- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe56006c980 msgr2=0x7fe56006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe56006c980 0x7fe56006ee40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe564034000 tx=0x7fe56402e000 comp rx=0 tx=0).stop 2024-04-26T20:23:22.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 -- 172.21.15.37:0/1063927863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe57415d460 msgr2=0x7fe5740d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe57415d460 0x7fe5740d01f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe56c007e40 tx=0x7fe56c00e4b0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 -- 172.21.15.37:0/1063927863 shutdown_connections 2024-04-26T20:23:22.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe57415d460 0x7fe5740d01f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5740cd100 0x7fe5740cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe56006c980 0x7fe56006ee40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 --2- 172.21.15.37:0/1063927863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe574073c40 0x7fe5741f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 -- 172.21.15.37:0/1063927863 >> 172.21.15.37:0/1063927863 conn(0x7fe5740d0d70 msgr2=0x7fe5740b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:22.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 -- 172.21.15.37:0/1063927863 shutdown_connections 2024-04-26T20:23:22.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.680+0000 7fe57a19d640 1 -- 172.21.15.37:0/1063927863 wait complete. 2024-04-26T20:23:22.681 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:22.680+0000 7f9a8a1fb640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:22.681 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:22.680+0000 7f9a8a1fb640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:22.681 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:22.680+0000 7f9a8a1fb640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:22.682 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:22.680+0000 7f9a8a1fb640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:22.682 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:22.680+0000 7f9a8a1fb640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:22.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:22.692 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rmdir ./volumes/_nogroup/subvol19350/.snap/subvol_snap85010) 2024-04-26T20:23:22.695 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:22.694+0000 7f9a8b1fd640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:22.695 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:22.694+0000 7f9a8b1fd640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:22.695 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:22.694+0000 7f9a8b1fd640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:22.695 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:22.694+0000 7f9a8b1fd640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:22.695 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:22.694+0000 7f9a8b1fd640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:22.721 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone57050 2024-04-26T20:23:22.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.866+0000 7f5eaf0f1640 1 Processor -- start 2024-04-26T20:23:22.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.866+0000 7f5eaf0f1640 1 -- start start 2024-04-26T20:23:22.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.866+0000 7f5eaf0f1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5ea8073a00 0x7f5ea8167280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.866+0000 7f5eaf0f1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea8073de0 con 0x7f5ea806df90 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.866+0000 7f5eaf0f1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea80ccaf0 con 0x7f5ea815c450 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.866+0000 7f5eaf0f1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5ea815c830 con 0x7f5ea8073a00 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5ea806df90 0x7f5ea80cc3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58984/0 (socket says 172.21.15.37:58984) 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea7fff640 1 -- 172.21.15.37:0/2115009228 learned_addr learned my addr 172.21.15.37:0/2115009228 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ead68e640 1 --2- 172.21.15.37:0/2115009228 >> v2:172.21.15.134:3300/0 conn(0x7f5ea8073a00 0x7f5ea8167280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638196535 0 0) 0x7f5e90001110 con 0x7f5ea806df90 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e94003520 con 0x7f5ea806df90 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4190214140 0 0) 0x7f5e94003520 con 0x7f5ea806df90 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 >> v2:172.21.15.134:3300/0 conn(0x7f5ea8073a00 msgr2=0x7f5ea8167280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 --2- 172.21.15.37:0/2115009228 >> v2:172.21.15.134:3300/0 conn(0x7f5ea8073a00 0x7f5ea8167280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 >> v1:172.21.15.134:6789/0 conn(0x7f5ea815c450 legacy=0x7f5ea80cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea815d600 con 0x7f5ea806df90 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f5e90001840 con 0x7f5ea806df90 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 >> v1:172.21.15.37:6789/0 conn(0x7f5ea806df90 legacy=0x7f5ea80cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 --2- 172.21.15.37:0/2115009228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e940039b0 0x7f5e94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 --2- 172.21.15.37:0/2115009228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 0x7f5e94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 --2- 172.21.15.37:0/2115009228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e940051c0 0x7f5e9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea815c830 con 0x7f5e94004800 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9400c160 con 0x7f5e940039b0 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9400c360 con 0x7f5e940051c0 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5ea7fff640 1 --2- 172.21.15.37:0/2115009228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 0x7f5e94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.867+0000 7f5eace8d640 1 --2- 172.21.15.37:0/2115009228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e940039b0 0x7f5e94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.868+0000 7f5ead68e640 1 --2- 172.21.15.37:0/2115009228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e940051c0 0x7f5e9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.868+0000 7f5ea7fff640 1 -- 172.21.15.37:0/2115009228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e940051c0 msgr2=0x7f5e9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.868+0000 7f5ea7fff640 1 --2- 172.21.15.37:0/2115009228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e940051c0 0x7f5e9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.868+0000 7f5ea7fff640 1 -- 172.21.15.37:0/2115009228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e940039b0 msgr2=0x7f5e94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:22.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.868+0000 7f5ea7fff640 1 --2- 172.21.15.37:0/2115009228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e940039b0 0x7f5e94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.868+0000 7f5ead68e640 1 -- 172.21.15.37:0/2115009228 reap_dead start 2024-04-26T20:23:22.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.868+0000 7f5ea7fff640 1 -- 172.21.15.37:0/2115009228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e94003520 con 0x7f5e94004800 2024-04-26T20:23:22.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.868+0000 7f5ea7fff640 1 --2- 172.21.15.37:0/2115009228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 0x7f5e94004c80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5e90007ec0 tx=0x7f5e94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cf2e88b2b5a1c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:22.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.869+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000c0e0 con 0x7f5e94004800 2024-04-26T20:23:22.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.869+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e90005af0 con 0x7f5e94004800 2024-04-26T20:23:22.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.869+0000 7f5ea77fe640 1 -- 172.21.15.37:0/2115009228 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90007070 con 0x7f5e94004800 2024-04-26T20:23:22.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.869+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/2115009228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 msgr2=0x7f5e94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.869+0000 7f5eaf0f1640 1 --2- 172.21.15.37:0/2115009228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 0x7f5e94004c80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5e90007ec0 tx=0x7f5e94005830 comp rx=0 tx=0).stop 2024-04-26T20:23:22.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.869+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/2115009228 shutdown_connections 2024-04-26T20:23:22.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.869+0000 7f5eaf0f1640 1 --2- 172.21.15.37:0/2115009228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 0x7f5e94004c80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.869+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/2115009228 >> 172.21.15.37:0/2115009228 conn(0x7f5ea80d0d70 msgr2=0x7f5ea80d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:22.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.869+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/2115009228 shutdown_connections 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.869+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/2115009228 wait complete. 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.870+0000 7f5eaf0f1640 1 Processor -- start 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.870+0000 7f5eaf0f1640 1 -- start start 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.871+0000 7f5eaf0f1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 0x7f5ea809cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.871+0000 7f5eaf0f1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea80c8850 0x7f5ea809daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.871+0000 7f5eaf0f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8150fa0 0x7f5ea81513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.871+0000 7f5eaf0f1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea806fb10 con 0x7f5e94004800 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.871+0000 7f5eaf0f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea806f830 con 0x7f5ea8150fa0 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.871+0000 7f5eaf0f1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea80cbcd0 con 0x7f5ea80c8850 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.871+0000 7f5eace8d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 0x7f5ea809cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.871+0000 7f5eace8d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 0x7f5ea809cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51624/0 (socket says 172.21.15.37:51624) 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.871+0000 7f5eace8d640 1 -- 172.21.15.37:0/3462506589 learned_addr learned my addr 172.21.15.37:0/3462506589 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5ea7fff640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea80c8850 0x7f5ea809daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5ead68e640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8150fa0 0x7f5ea81513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5eace8d640 1 -- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea80c8850 msgr2=0x7f5ea809daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5eace8d640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea80c8850 0x7f5ea809daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5eace8d640 1 -- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8150fa0 msgr2=0x7f5ea81513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5eace8d640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8150fa0 0x7f5ea81513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5eace8d640 1 -- 172.21.15.37:0/3462506589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea80ca060 con 0x7f5e94004800 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5eace8d640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 0x7f5ea809cfb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000f920 tx=0x7f5ea000fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5ea57fa640 1 -- 172.21.15.37:0/3462506589 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0003b80 con 0x7f5e94004800 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5ea57fa640 1 -- 172.21.15.37:0/3462506589 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea0018070 con 0x7f5e94004800 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/3462506589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea8151920 con 0x7f5e94004800 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5ea57fa640 1 -- 172.21.15.37:0/3462506589 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000d7f0 con 0x7f5e94004800 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.872+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/3462506589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea80c92a0 con 0x7f5e94004800 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.873+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/3462506589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea806a360 con 0x7f5e94004800 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.874+0000 7f5ea57fa640 1 -- 172.21.15.37:0/3462506589 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5ea000d990 con 0x7f5e94004800 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.874+0000 7f5ea57fa640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5e8806c980 0x7f5e8806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.874+0000 7f5ea57fa640 1 -- 172.21.15.37:0/3462506589 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5ea008be30 con 0x7f5e94004800 2024-04-26T20:23:22.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.874+0000 7f5ea7fff640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5e8806c980 0x7f5e8806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.875+0000 7f5ea7fff640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5e8806c980 0x7f5e8806ee40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000c490 tx=0x7f5e90030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:22.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:22.878+0000 7f5ea57fa640 1 -- 172.21.15.37:0/3462506589 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ea005a1e0 con 0x7f5e94004800 2024-04-26T20:23:23.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.101+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/3462506589 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone57050", "target": ["mon-mgr", ""]}) -- 0x7f5ea8075670 con 0x7f5e8806c980 2024-04-26T20:23:23.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.107+0000 7f5ea57fa640 1 -- 172.21.15.37:0/3462506589 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+165 (secure 0 0 0) 0x7f5ea8075670 con 0x7f5e8806c980 2024-04-26T20:23:23.108 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:23:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "status": { 2024-04-26T20:23:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "pending", 2024-04-26T20:23:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "source": { 2024-04-26T20:23:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "volume": "cephfs", 2024-04-26T20:23:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "subvolume": "subvol19350", 2024-04-26T20:23:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot": "subvol_snap85010" 2024-04-26T20:23:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:23.108 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:23:23.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5e8806c980 msgr2=0x7f5e8806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:23.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5e8806c980 0x7f5e8806ee40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000c490 tx=0x7f5e90030000 comp rx=0 tx=0).stop 2024-04-26T20:23:23.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/3462506589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 msgr2=0x7f5ea809cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:23.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 0x7f5ea809cfb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000f920 tx=0x7f5ea000fde0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/3462506589 shutdown_connections 2024-04-26T20:23:23.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8150fa0 0x7f5ea81513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5e8806c980 0x7f5e8806ee40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea80c8850 0x7f5ea809daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 --2- 172.21.15.37:0/3462506589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e94004800 0x7f5ea809cfb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/3462506589 >> 172.21.15.37:0/3462506589 conn(0x7f5ea80d0d70 msgr2=0x7f5ea815c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:23.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/3462506589 shutdown_connections 2024-04-26T20:23:23.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:23.110+0000 7f5eaf0f1640 1 -- 172.21.15.37:0/3462506589 wait complete. 2024-04-26T20:23:24.121 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone57050 2024-04-26T20:23:24.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.238+0000 7f2bed186640 1 Processor -- start 2024-04-26T20:23:24.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.238+0000 7f2bed186640 1 -- start start 2024-04-26T20:23:24.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2bed186640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2be815f470 0x7f2be80d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:24.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2bed186640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2be80d0d30 0x7f2be80d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:24.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2bed186640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be80d2ed0 con 0x7f2be80d1fe0 2024-04-26T20:23:24.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2bed186640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2be80d1190 con 0x7f2be80d0d30 2024-04-26T20:23:24.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2bed186640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2be80d31b0 con 0x7f2be815f470 2024-04-26T20:23:24.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2be659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2be80d0d30 0x7f2be80d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:24.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2be759e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2be80d1fe0 0x7f2be8162fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58998/0 (socket says 172.21.15.37:58998) 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2be659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2be80d0d30 0x7f2be80d1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51628/0 (socket says 172.21.15.37:51628) 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2be759e640 1 -- 172.21.15.37:0/997428158 learned_addr learned my addr 172.21.15.37:0/997428158 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2be6d9d640 1 --2- 172.21.15.37:0/997428158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2be815f470 0x7f2be80d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2be659c640 1 -- 172.21.15.37:0/997428158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2be815f470 msgr2=0x7f2be80d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2be659c640 1 --2- 172.21.15.37:0/997428158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2be815f470 0x7f2be80d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2be659c640 1 -- 172.21.15.37:0/997428158 >> v1:172.21.15.37:6789/0 conn(0x7f2be80d1fe0 legacy=0x7f2be8162fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2be659c640 1 -- 172.21.15.37:0/997428158 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be80d3490 con 0x7f2be80d0d30 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.239+0000 7f2be5d9b640 1 -- 172.21.15.37:0/997428158 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794255560 0 0) 0x7f2be80d2ed0 con 0x7f2be80d1fe0 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.240+0000 7f2be659c640 1 --2- 172.21.15.37:0/997428158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be80d0d30 0x7f2be80d1aa0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0097d0 tx=0x7f2bdc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6594de4e0acbdcfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.240+0000 7f2be5d9b640 1 -- 172.21.15.37:0/997428158 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc037070 con 0x7f2be80d0d30 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.240+0000 7f2be5d9b640 1 -- 172.21.15.37:0/997428158 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bdc032440 con 0x7f2be80d0d30 2024-04-26T20:23:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.240+0000 7f2be5d9b640 1 -- 172.21.15.37:0/997428158 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc032810 con 0x7f2be80d0d30 2024-04-26T20:23:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.241+0000 7f2bed186640 1 -- 172.21.15.37:0/997428158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be80d0d30 msgr2=0x7f2be80d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.241+0000 7f2bed186640 1 --2- 172.21.15.37:0/997428158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be80d0d30 0x7f2be80d1aa0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0097d0 tx=0x7f2bdc02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:23:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.241+0000 7f2bed186640 1 -- 172.21.15.37:0/997428158 shutdown_connections 2024-04-26T20:23:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.241+0000 7f2bed186640 1 --2- 172.21.15.37:0/997428158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be80d0d30 0x7f2be80d1aa0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.241+0000 7f2bed186640 1 --2- 172.21.15.37:0/997428158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2be815f470 0x7f2be80d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.241+0000 7f2bed186640 1 -- 172.21.15.37:0/997428158 >> 172.21.15.37:0/997428158 conn(0x7f2be809e510 msgr2=0x7f2be80d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.241+0000 7f2bed186640 1 -- 172.21.15.37:0/997428158 shutdown_connections 2024-04-26T20:23:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.241+0000 7f2bed186640 1 -- 172.21.15.37:0/997428158 wait complete. 2024-04-26T20:23:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.242+0000 7f2bed186640 1 Processor -- start 2024-04-26T20:23:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.243+0000 7f2bed186640 1 -- start start 2024-04-26T20:23:24.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.243+0000 7f2bed186640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be80d0d30 0x7f2be8153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:24.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.243+0000 7f2bed186640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2be80d1fe0 0x7f2be80ba040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:24.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.243+0000 7f2bed186640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815f470 0x7f2be815ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:24.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.243+0000 7f2bed186640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2be815f850 con 0x7f2be815f470 2024-04-26T20:23:24.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.243+0000 7f2bed186640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2be80d3a50 con 0x7f2be80d0d30 2024-04-26T20:23:24.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.243+0000 7f2bed186640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2be80d3f70 con 0x7f2be80d1fe0 2024-04-26T20:23:24.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.243+0000 7f2be759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815f470 0x7f2be815ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.243+0000 7f2be759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815f470 0x7f2be815ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51632/0 (socket says 172.21.15.37:51632) 2024-04-26T20:23:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.243+0000 7f2be759e640 1 -- 172.21.15.37:0/1962718231 learned_addr learned my addr 172.21.15.37:0/1962718231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2be659c640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2be80d1fe0 0x7f2be80ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2be6d9d640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be80d0d30 0x7f2be8153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2be759e640 1 -- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2be80d1fe0 msgr2=0x7f2be80ba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2be759e640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2be80d1fe0 0x7f2be80ba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2be759e640 1 -- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be80d0d30 msgr2=0x7f2be8153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2be759e640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be80d0d30 0x7f2be8153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2be759e640 1 -- 172.21.15.37:0/1962718231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be8153960 con 0x7f2be815f470 2024-04-26T20:23:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2be759e640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815f470 0x7f2be815ba00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2be000c700 tx=0x7f2be000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/1962718231 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0018070 con 0x7f2be815f470 2024-04-26T20:23:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/1962718231 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be000ce10 con 0x7f2be815f470 2024-04-26T20:23:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2bed186640 1 -- 172.21.15.37:0/1962718231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be80ba580 con 0x7f2be815f470 2024-04-26T20:23:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/1962718231 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be000a860 con 0x7f2be815f470 2024-04-26T20:23:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.244+0000 7f2bed186640 1 -- 172.21.15.37:0/1962718231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be80b9560 con 0x7f2be815f470 2024-04-26T20:23:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.245+0000 7f2bed186640 1 -- 172.21.15.37:0/1962718231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be806a360 con 0x7f2be815f470 2024-04-26T20:23:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.246+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/1962718231 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f2be001b070 con 0x7f2be815f470 2024-04-26T20:23:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.246+0000 7f2bcf7fe640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd406cad0 0x7f2bd406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.246+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/1962718231 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2be008b290 con 0x7f2be815f470 2024-04-26T20:23:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.246+0000 7f2be6d9d640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd406cad0 0x7f2bd406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.247+0000 7f2be6d9d640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd406cad0 0x7f2bd406ef90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8006fd0 tx=0x7f2bd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.250+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/1962718231 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2be0058e80 con 0x7f2be815f470 2024-04-26T20:23:24.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.475+0000 7f2bed186640 1 -- 172.21.15.37:0/1962718231 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone57050", "target": ["mon-mgr", ""]}) -- 0x7f2be80d5c30 con 0x7f2bd406cad0 2024-04-26T20:23:24.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.481+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/1962718231 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+165 (secure 0 0 0) 0x7f2be80d5c30 con 0x7f2bd406cad0 2024-04-26T20:23:24.482 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:23:24.482 INFO:teuthology.orchestra.run.smithi037.stdout: "status": { 2024-04-26T20:23:24.482 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "pending", 2024-04-26T20:23:24.482 INFO:teuthology.orchestra.run.smithi037.stdout: "source": { 2024-04-26T20:23:24.482 INFO:teuthology.orchestra.run.smithi037.stdout: "volume": "cephfs", 2024-04-26T20:23:24.482 INFO:teuthology.orchestra.run.smithi037.stdout: "subvolume": "subvol19350", 2024-04-26T20:23:24.482 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot": "subvol_snap85010" 2024-04-26T20:23:24.482 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:24.482 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:24.482 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.483+0000 7f2bed186640 1 -- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd406cad0 msgr2=0x7f2bd406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.483+0000 7f2bed186640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd406cad0 0x7f2bd406ef90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8006fd0 tx=0x7f2bd8008040 comp rx=0 tx=0).stop 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.483+0000 7f2bed186640 1 -- 172.21.15.37:0/1962718231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815f470 msgr2=0x7f2be815ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.483+0000 7f2bed186640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815f470 0x7f2be815ba00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2be000c700 tx=0x7f2be000cbc0 comp rx=0 tx=0).stop 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.483+0000 7f2bed186640 1 -- 172.21.15.37:0/1962718231 shutdown_connections 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.483+0000 7f2bed186640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815f470 0x7f2be815ba00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.483+0000 7f2bed186640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2bd406cad0 0x7f2bd406ef90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.483+0000 7f2bed186640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2be80d1fe0 0x7f2be80ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.483+0000 7f2bed186640 1 --2- 172.21.15.37:0/1962718231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be80d0d30 0x7f2be8153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.483+0000 7f2bed186640 1 -- 172.21.15.37:0/1962718231 >> 172.21.15.37:0/1962718231 conn(0x7f2be809e510 msgr2=0x7f2be80d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.484+0000 7f2bed186640 1 -- 172.21.15.37:0/1962718231 shutdown_connections 2024-04-26T20:23:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:24.484+0000 7f2bed186640 1 -- 172.21.15.37:0/1962718231 wait complete. 2024-04-26T20:23:25.495 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone57050 2024-04-26T20:23:25.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.612+0000 7fdd7e36a640 1 Processor -- start 2024-04-26T20:23:25.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.612+0000 7fdd7e36a640 1 -- start start 2024-04-26T20:23:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.613+0000 7fdd7e36a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdd780d1990 0x7fdd780ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.613+0000 7fdd7e36a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd78074050 con 0x7fdd7806c950 2024-04-26T20:23:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.613+0000 7fdd7e36a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd780b1060 con 0x7fdd7815f3c0 2024-04-26T20:23:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.613+0000 7fdd7e36a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fdd780b1340 con 0x7fdd780d1990 2024-04-26T20:23:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.613+0000 7fdd7c907640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdd780d1990 0x7fdd780ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.613+0000 7fdd777fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdd7806c950 0x7fdd780cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59012/0 (socket says 172.21.15.37:59012) 2024-04-26T20:23:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.613+0000 7fdd777fe640 1 -- 172.21.15.37:0/3141962135 learned_addr learned my addr 172.21.15.37:0/3141962135 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.614+0000 7fdd7c907640 1 -- 172.21.15.37:0/3141962135 >> v1:172.21.15.134:6789/0 conn(0x7fdd7815f3c0 legacy=0x7fdd7806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.614+0000 7fdd7c907640 1 -- 172.21.15.37:0/3141962135 >> v1:172.21.15.37:6789/0 conn(0x7fdd7806c950 legacy=0x7fdd780cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.614+0000 7fdd76ffd640 1 -- 172.21.15.37:0/3141962135 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 175810555 0 0) 0x7fdd78074050 con 0x7fdd7806c950 2024-04-26T20:23:25.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.614+0000 7fdd7c907640 1 -- 172.21.15.37:0/3141962135 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd780b1620 con 0x7fdd780d1990 2024-04-26T20:23:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.614+0000 7fdd7c907640 1 --2- 172.21.15.37:0/3141962135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd780d1990 0x7fdd780ccbe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009d90 tx=0x7fdd7002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49c9685f64d6ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.615+0000 7fdd76ffd640 1 -- 172.21.15.37:0/3141962135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70031600 con 0x7fdd780d1990 2024-04-26T20:23:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.615+0000 7fdd76ffd640 1 -- 172.21.15.37:0/3141962135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd700317a0 con 0x7fdd780d1990 2024-04-26T20:23:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.615+0000 7fdd76ffd640 1 -- 172.21.15.37:0/3141962135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70031b70 con 0x7fdd780d1990 2024-04-26T20:23:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.615+0000 7fdd7e36a640 1 -- 172.21.15.37:0/3141962135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd780d1990 msgr2=0x7fdd780ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.615+0000 7fdd7e36a640 1 --2- 172.21.15.37:0/3141962135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd780d1990 0x7fdd780ccbe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009d90 tx=0x7fdd7002a610 comp rx=0 tx=0).stop 2024-04-26T20:23:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.615+0000 7fdd7e36a640 1 -- 172.21.15.37:0/3141962135 shutdown_connections 2024-04-26T20:23:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.615+0000 7fdd7e36a640 1 --2- 172.21.15.37:0/3141962135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd780d1990 0x7fdd780ccbe0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.615+0000 7fdd7e36a640 1 -- 172.21.15.37:0/3141962135 >> 172.21.15.37:0/3141962135 conn(0x7fdd780d1e80 msgr2=0x7fdd780d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.615+0000 7fdd7e36a640 1 -- 172.21.15.37:0/3141962135 shutdown_connections 2024-04-26T20:23:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.615+0000 7fdd7e36a640 1 -- 172.21.15.37:0/3141962135 wait complete. 2024-04-26T20:23:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.616+0000 7fdd7e36a640 1 Processor -- start 2024-04-26T20:23:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.617+0000 7fdd7e36a640 1 -- start start 2024-04-26T20:23:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.617+0000 7fdd7e36a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd7806c950 0x7fdd781e4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.617+0000 7fdd7e36a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd780d1990 0x7fdd780c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.617+0000 7fdd7e36a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd7815f3c0 0x7fdd7814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.617+0000 7fdd7e36a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd780d1530 con 0x7fdd7815f3c0 2024-04-26T20:23:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.617+0000 7fdd7e36a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd780d13b0 con 0x7fdd780d1990 2024-04-26T20:23:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.617+0000 7fdd7e36a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7815f7a0 con 0x7fdd7806c950 2024-04-26T20:23:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.617+0000 7fdd7c907640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd7815f3c0 0x7fdd7814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd7c907640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd7815f3c0 0x7fdd7814fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51660/0 (socket says 172.21.15.37:51660) 2024-04-26T20:23:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd7c907640 1 -- 172.21.15.37:0/1782537788 learned_addr learned my addr 172.21.15.37:0/1782537788 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd777fe640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd780d1990 0x7fdd780c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd77fff640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd7806c950 0x7fdd781e4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd7c907640 1 -- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd7806c950 msgr2=0x7fdd781e4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd7c907640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd7806c950 0x7fdd781e4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd7c907640 1 -- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd780d1990 msgr2=0x7fdd780c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd7c907640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd780d1990 0x7fdd780c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd7c907640 1 -- 172.21.15.37:0/1782537788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd780c7fd0 con 0x7fdd7815f3c0 2024-04-26T20:23:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd7c907640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd7815f3c0 0x7fdd7814fef0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009eb0 tx=0x7fdd70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd74ff9640 1 -- 172.21.15.37:0/1782537788 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70040070 con 0x7fdd7815f3c0 2024-04-26T20:23:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd74ff9640 1 -- 172.21.15.37:0/1782537788 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd70030b60 con 0x7fdd7815f3c0 2024-04-26T20:23:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd7e36a640 1 -- 172.21.15.37:0/1782537788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd780c82b0 con 0x7fdd7815f3c0 2024-04-26T20:23:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd74ff9640 1 -- 172.21.15.37:0/1782537788 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70045460 con 0x7fdd7815f3c0 2024-04-26T20:23:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.618+0000 7fdd7e36a640 1 -- 172.21.15.37:0/1782537788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd781e6b20 con 0x7fdd7815f3c0 2024-04-26T20:23:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.620+0000 7fdd7e36a640 1 -- 172.21.15.37:0/1782537788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd780c8480 con 0x7fdd7815f3c0 2024-04-26T20:23:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.620+0000 7fdd74ff9640 1 -- 172.21.15.37:0/1782537788 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fdd70039040 con 0x7fdd7815f3c0 2024-04-26T20:23:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.620+0000 7fdd74ff9640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd6406cad0 0x7fdd6406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.620+0000 7fdd74ff9640 1 -- 172.21.15.37:0/1782537788 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdd700ad840 con 0x7fdd7815f3c0 2024-04-26T20:23:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.621+0000 7fdd77fff640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd6406cad0 0x7fdd6406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.621+0000 7fdd77fff640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd6406cad0 0x7fdd6406ef90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006fd0 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.625+0000 7fdd74ff9640 1 -- 172.21.15.37:0/1782537788 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd7007b9a0 con 0x7fdd7815f3c0 2024-04-26T20:23:25.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.848+0000 7fdd7e36a640 1 -- 172.21.15.37:0/1782537788 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone57050", "target": ["mon-mgr", ""]}) -- 0x7fdd7806a360 con 0x7fdd6406cad0 2024-04-26T20:23:25.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.855+0000 7fdd74ff9640 1 -- 172.21.15.37:0/1782537788 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+165 (secure 0 0 0) 0x7fdd7806a360 con 0x7fdd6406cad0 2024-04-26T20:23:25.855 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:23:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "status": { 2024-04-26T20:23:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "pending", 2024-04-26T20:23:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "source": { 2024-04-26T20:23:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "volume": "cephfs", 2024-04-26T20:23:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "subvolume": "subvol19350", 2024-04-26T20:23:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot": "subvol_snap85010" 2024-04-26T20:23:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:25.855 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:23:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 -- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd6406cad0 msgr2=0x7fdd6406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd6406cad0 0x7fdd6406ef90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006fd0 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2024-04-26T20:23:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 -- 172.21.15.37:0/1782537788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd7815f3c0 msgr2=0x7fdd7814fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd7815f3c0 0x7fdd7814fef0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009eb0 tx=0x7fdd70004a40 comp rx=0 tx=0).stop 2024-04-26T20:23:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 -- 172.21.15.37:0/1782537788 shutdown_connections 2024-04-26T20:23:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd7815f3c0 0x7fdd7814fef0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd780d1990 0x7fdd780c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:25.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdd6406cad0 0x7fdd6406ef90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:25.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 --2- 172.21.15.37:0/1782537788 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd7806c950 0x7fdd781e4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:25.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 -- 172.21.15.37:0/1782537788 >> 172.21.15.37:0/1782537788 conn(0x7fdd780d1e80 msgr2=0x7fdd780b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:25.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 -- 172.21.15.37:0/1782537788 shutdown_connections 2024-04-26T20:23:25.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:25.857+0000 7fdd7e36a640 1 -- 172.21.15.37:0/1782537788 wait complete. 2024-04-26T20:23:26.868 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone57050 2024-04-26T20:23:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.982+0000 7f5bbf65b640 1 Processor -- start 2024-04-26T20:23:26.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.982+0000 7f5bbf65b640 1 -- start start 2024-04-26T20:23:26.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.983+0000 7f5bbf65b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bb815f470 0x7f5bb80d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:26.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.983+0000 7f5bbf65b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5bb80d21b0 0x7f5bb80b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:26.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.983+0000 7f5bbf65b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bb80d2ed0 con 0x7f5bb80d0d30 2024-04-26T20:23:26.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.983+0000 7f5bbf65b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f5bb80d1190 con 0x7f5bb80d21b0 2024-04-26T20:23:26.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.983+0000 7f5bbf65b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb80d31b0 con 0x7f5bb815f470 2024-04-26T20:23:26.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.983+0000 7f5bbdbf8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5bb80d21b0 0x7f5bb80b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:26.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.984+0000 7f5bbdbf8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5bb80d21b0 0x7f5bb80b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53298/0 (socket says 172.21.15.37:53298) 2024-04-26T20:23:26.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.984+0000 7f5bbdbf8640 1 -- 172.21.15.37:0/112575641 learned_addr learned my addr 172.21.15.37:0/112575641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:26.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.984+0000 7f5bbd3f7640 1 --2- 172.21.15.37:0/112575641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bb815f470 0x7f5bb80d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:26.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.984+0000 7f5bbdbf8640 1 -- 172.21.15.37:0/112575641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bb815f470 msgr2=0x7f5bb80d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:26.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.984+0000 7f5bbdbf8640 1 --2- 172.21.15.37:0/112575641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bb815f470 0x7f5bb80d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:26.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.984+0000 7f5bbdbf8640 1 -- 172.21.15.37:0/112575641 >> v1:172.21.15.134:6789/0 conn(0x7f5bb80d0d30 legacy=0x7f5bb80d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:26.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.984+0000 7f5bbdbf8640 1 -- 172.21.15.37:0/112575641 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb80d3490 con 0x7f5bb80d21b0 2024-04-26T20:23:26.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5bbdbf8640 1 --2- 172.21.15.37:0/112575641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb80d21b0 0x7f5bb80b7fb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009e50 tx=0x7f5bb002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5661008d1b53aef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:26.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5baffff640 1 -- 172.21.15.37:0/112575641 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0036070 con 0x7f5bb80d21b0 2024-04-26T20:23:26.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5baffff640 1 -- 172.21.15.37:0/112575641 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb0031600 con 0x7f5bb80d21b0 2024-04-26T20:23:26.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5baffff640 1 -- 172.21.15.37:0/112575641 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00319d0 con 0x7f5bb80d21b0 2024-04-26T20:23:26.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5bbf65b640 1 -- 172.21.15.37:0/112575641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb80d21b0 msgr2=0x7f5bb80b7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:26.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5bbf65b640 1 --2- 172.21.15.37:0/112575641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb80d21b0 0x7f5bb80b7fb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009e50 tx=0x7f5bb002a6c0 comp rx=0 tx=0).stop 2024-04-26T20:23:26.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5bbf65b640 1 -- 172.21.15.37:0/112575641 shutdown_connections 2024-04-26T20:23:26.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5bbf65b640 1 --2- 172.21.15.37:0/112575641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb80d21b0 0x7f5bb80b7fb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:26.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5bbf65b640 1 --2- 172.21.15.37:0/112575641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bb815f470 0x7f5bb80d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:26.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5bbf65b640 1 -- 172.21.15.37:0/112575641 >> 172.21.15.37:0/112575641 conn(0x7f5bb809e510 msgr2=0x7f5bb80d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:26.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5bbf65b640 1 -- 172.21.15.37:0/112575641 shutdown_connections 2024-04-26T20:23:26.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.985+0000 7f5bbf65b640 1 -- 172.21.15.37:0/112575641 wait complete. 2024-04-26T20:23:26.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.987+0000 7f5bbf65b640 1 Processor -- start 2024-04-26T20:23:26.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.987+0000 7f5bbf65b640 1 -- start start 2024-04-26T20:23:26.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.987+0000 7f5bbf65b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb80d0d30 0x7f5bb8151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:26.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.987+0000 7f5bbf65b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bb80d21b0 0x7f5bb80ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:26.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.987+0000 7f5bbf65b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bb815f470 0x7f5bb80ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:26.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.987+0000 7f5bbf65b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb815f850 con 0x7f5bb80d0d30 2024-04-26T20:23:26.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.987+0000 7f5bbf65b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb80d3a50 con 0x7f5bb80d21b0 2024-04-26T20:23:26.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.987+0000 7f5bbf65b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb80d3f70 con 0x7f5bb815f470 2024-04-26T20:23:26.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.987+0000 7f5bbd3f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb80d0d30 0x7f5bb8151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:26.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.987+0000 7f5bbd3f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb80d0d30 0x7f5bb8151170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53306/0 (socket says 172.21.15.37:53306) 2024-04-26T20:23:26.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.987+0000 7f5bbd3f7640 1 -- 172.21.15.37:0/4247522413 learned_addr learned my addr 172.21.15.37:0/4247522413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5bbdbf8640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bb815f470 0x7f5bb80ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5bbcbf6640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bb80d21b0 0x7f5bb80ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5bbd3f7640 1 -- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bb815f470 msgr2=0x7f5bb80ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5bbd3f7640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bb815f470 0x7f5bb80ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5bbd3f7640 1 -- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bb80d21b0 msgr2=0x7f5bb80ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5bbd3f7640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bb80d21b0 0x7f5bb80ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5bbd3f7640 1 -- 172.21.15.37:0/4247522413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb81ea5a0 con 0x7f5bb80d0d30 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5bbd3f7640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb80d0d30 0x7f5bb8151170 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400d620 tx=0x7f5bb400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5badffb640 1 -- 172.21.15.37:0/4247522413 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400b620 con 0x7f5bb80d0d30 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5badffb640 1 -- 172.21.15.37:0/4247522413 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb4019070 con 0x7f5bb80d0d30 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5badffb640 1 -- 172.21.15.37:0/4247522413 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400ba00 con 0x7f5bb80d0d30 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5bbf65b640 1 -- 172.21.15.37:0/4247522413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb80b86b0 con 0x7f5bb80d0d30 2024-04-26T20:23:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.988+0000 7f5bbf65b640 1 -- 172.21.15.37:0/4247522413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb80b8360 con 0x7f5bb80d0d30 2024-04-26T20:23:26.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.990+0000 7f5bbf65b640 1 -- 172.21.15.37:0/4247522413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb806a360 con 0x7f5bb80d0d30 2024-04-26T20:23:26.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.990+0000 7f5badffb640 1 -- 172.21.15.37:0/4247522413 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f5bb400bba0 con 0x7f5bb80d0d30 2024-04-26T20:23:26.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.990+0000 7f5badffb640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5b9c06caa0 0x7f5b9c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:26.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.991+0000 7f5badffb640 1 -- 172.21.15.37:0/4247522413 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5bb408c0c0 con 0x7f5bb80d0d30 2024-04-26T20:23:26.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.991+0000 7f5bbcbf6640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5b9c06caa0 0x7f5b9c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:26.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.991+0000 7f5bbcbf6640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5b9c06caa0 0x7f5b9c06ef60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8006fd0 tx=0x7f5ba8009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:26.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:26.995+0000 7f5badffb640 1 -- 172.21.15.37:0/4247522413 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5bb4059cb0 con 0x7f5bb80d0d30 2024-04-26T20:23:27.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.218+0000 7f5bbf65b640 1 -- 172.21.15.37:0/4247522413 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone57050", "target": ["mon-mgr", ""]}) -- 0x7f5bb80d1d10 con 0x7f5b9c06caa0 2024-04-26T20:23:27.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.225+0000 7f5badffb640 1 -- 172.21.15.37:0/4247522413 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+165 (secure 0 0 0) 0x7f5bb80d1d10 con 0x7f5b9c06caa0 2024-04-26T20:23:27.227 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:23:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: "status": { 2024-04-26T20:23:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "pending", 2024-04-26T20:23:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: "source": { 2024-04-26T20:23:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: "volume": "cephfs", 2024-04-26T20:23:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: "subvolume": "subvol19350", 2024-04-26T20:23:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot": "subvol_snap85010" 2024-04-26T20:23:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:27.227 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:23:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 -- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5b9c06caa0 msgr2=0x7f5b9c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5b9c06caa0 0x7f5b9c06ef60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8006fd0 tx=0x7f5ba8009060 comp rx=0 tx=0).stop 2024-04-26T20:23:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 -- 172.21.15.37:0/4247522413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb80d0d30 msgr2=0x7f5bb8151170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb80d0d30 0x7f5bb8151170 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400d620 tx=0x7f5bb400dae0 comp rx=0 tx=0).stop 2024-04-26T20:23:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 -- 172.21.15.37:0/4247522413 shutdown_connections 2024-04-26T20:23:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5b9c06caa0 0x7f5b9c06ef60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bb815f470 0x7f5bb80ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bb80d21b0 0x7f5bb80ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 --2- 172.21.15.37:0/4247522413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb80d0d30 0x7f5bb8151170 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 -- 172.21.15.37:0/4247522413 >> 172.21.15.37:0/4247522413 conn(0x7f5bb809e510 msgr2=0x7f5bb80d5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:27.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 -- 172.21.15.37:0/4247522413 shutdown_connections 2024-04-26T20:23:27.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:27.228+0000 7f5bbf65b640 1 -- 172.21.15.37:0/4247522413 wait complete. 2024-04-26T20:23:28.240 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone57050 2024-04-26T20:23:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.355+0000 7f27548da640 1 Processor -- start 2024-04-26T20:23:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.355+0000 7f27548da640 1 -- start start 2024-04-26T20:23:28.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.356+0000 7f27548da640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27500b2930 0x7f27500d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.356+0000 7f27548da640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27500cd640 con 0x7f27500cff80 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.356+0000 7f27548da640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27500cd920 con 0x7f27500b2290 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.356+0000 7f27548da640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27500cdc00 con 0x7f27500b2930 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274dd9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f27500cff80 0x7f27500d1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33350/0 (socket says 172.21.15.37:33350) 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274dd9b640 1 -- 172.21.15.37:0/3638005844 learned_addr learned my addr 172.21.15.37:0/3638005844 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274ed9d640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27500b2930 0x7f27500d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3324104284 0 0) 0x7f27500cd640 con 0x7f27500cff80 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f273c003520 con 0x7f27500cff80 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 749026993 0 0) 0x7f27500cd920 con 0x7f27500b2290 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27500cd640 con 0x7f27500b2290 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 804372553 0 0) 0x7f273c003520 con 0x7f27500cff80 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27500b2930 msgr2=0x7f27500d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27500b2930 0x7f27500d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 >> v1:172.21.15.134:6789/0 conn(0x7f27500b2290 legacy=0x7f27500cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27500cdee0 con 0x7f27500cff80 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f2738001670 con 0x7f27500cff80 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 >> v1:172.21.15.37:6789/0 conn(0x7f27500cff80 legacy=0x7f27500d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c003b70 0x7f273c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27500b2930 0x7f273c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f273c004d40 0x7f273c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27500cdc00 con 0x7f273c004d40 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f273c00ba70 con 0x7f273c003b70 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f273c00bc20 con 0x7f27500b2930 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274e59c640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c003b70 0x7f273c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274ed9d640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f273c004d40 0x7f273c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.357+0000 7f274dd9b640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27500b2930 0x7f273c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.358+0000 7f274e59c640 1 -- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27500b2930 msgr2=0x7f273c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.358+0000 7f274e59c640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27500b2930 0x7f273c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.358+0000 7f274e59c640 1 -- 172.21.15.37:0/3638005844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f273c004d40 msgr2=0x7f273c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:28.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.358+0000 7f274e59c640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f273c004d40 0x7f273c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:28.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.358+0000 7f274e59c640 1 -- 172.21.15.37:0/3638005844 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c003520 con 0x7f273c003b70 2024-04-26T20:23:28.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.358+0000 7f274ed9d640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f273c004d40 0x7f273c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:28.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.358+0000 7f274e59c640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c003b70 0x7f273c003f70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27480018d0 tx=0x7f273c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=723287edde2bd22e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:28.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.359+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27480014f0 con 0x7f273c003b70 2024-04-26T20:23:28.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.359+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27480052d0 con 0x7f273c003b70 2024-04-26T20:23:28.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.359+0000 7f274d59a640 1 -- 172.21.15.37:0/3638005844 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274802aca0 con 0x7f273c003b70 2024-04-26T20:23:28.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.359+0000 7f27548da640 1 -- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c003b70 msgr2=0x7f273c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:28.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.359+0000 7f27548da640 1 --2- 172.21.15.37:0/3638005844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c003b70 0x7f273c003f70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27480018d0 tx=0x7f273c005830 comp rx=0 tx=0).stop 2024-04-26T20:23:28.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.359+0000 7f274ed9d640 1 -- 172.21.15.37:0/3638005844 reap_dead start 2024-04-26T20:23:28.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.359+0000 7f27548da640 1 -- 172.21.15.37:0/3638005844 shutdown_connections 2024-04-26T20:23:28.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.359+0000 7f27548da640 1 -- 172.21.15.37:0/3638005844 >> 172.21.15.37:0/3638005844 conn(0x7f27500d2ff0 msgr2=0x7f27500d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:28.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.359+0000 7f27548da640 1 -- 172.21.15.37:0/3638005844 shutdown_connections 2024-04-26T20:23:28.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.359+0000 7f27548da640 1 -- 172.21.15.37:0/3638005844 wait complete. 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.361+0000 7f27548da640 1 Processor -- start 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.361+0000 7f27548da640 1 -- start start 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f27548da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c003b70 0x7f275015ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f27548da640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275015aa80 0x7f27500b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f27548da640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2750150f60 0x7f27501513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f27548da640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27500d26f0 con 0x7f275015aa80 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f27548da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27500b2670 con 0x7f273c003b70 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f27548da640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27500d2870 con 0x7f2750150f60 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f274dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275015aa80 0x7f27500b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f274dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275015aa80 0x7f27500b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53324/0 (socket says 172.21.15.37:53324) 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f274dd9b640 1 -- 172.21.15.37:0/2589217815 learned_addr learned my addr 172.21.15.37:0/2589217815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f274ed9d640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2750150f60 0x7f27501513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f274e59c640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c003b70 0x7f275015ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f274dd9b640 1 -- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2750150f60 msgr2=0x7f27501513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f274dd9b640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2750150f60 0x7f27501513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f274dd9b640 1 -- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c003b70 msgr2=0x7f275015ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f274dd9b640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c003b70 0x7f275015ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f274dd9b640 1 -- 172.21.15.37:0/2589217815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2750161be0 con 0x7f275015aa80 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f274dd9b640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275015aa80 0x7f27500b3a60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2738007370 tx=0x7f2738007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f2746ffd640 1 -- 172.21.15.37:0/2589217815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273801a070 con 0x7f275015aa80 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.362+0000 7f2746ffd640 1 -- 172.21.15.37:0/2589217815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273800f940 con 0x7f275015aa80 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.363+0000 7f27548da640 1 -- 172.21.15.37:0/2589217815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27500b5ee0 con 0x7f275015aa80 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.363+0000 7f2746ffd640 1 -- 172.21.15.37:0/2589217815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273800fcf0 con 0x7f275015aa80 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.363+0000 7f27548da640 1 -- 172.21.15.37:0/2589217815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27500b64a0 con 0x7f275015aa80 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.364+0000 7f27548da640 1 -- 172.21.15.37:0/2589217815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f275006a360 con 0x7f275015aa80 2024-04-26T20:23:28.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.364+0000 7f2746ffd640 1 -- 172.21.15.37:0/2589217815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f273801d070 con 0x7f275015aa80 2024-04-26T20:23:28.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.365+0000 7f2746ffd640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273006caa0 0x7f273006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:28.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.365+0000 7f2746ffd640 1 -- 172.21.15.37:0/2589217815 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f273808b900 con 0x7f275015aa80 2024-04-26T20:23:28.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.365+0000 7f274e59c640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273006caa0 0x7f273006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.365+0000 7f274e59c640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273006caa0 0x7f273006ef60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2748032000 tx=0x7f2748009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:28.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.369+0000 7f2746ffd640 1 -- 172.21.15.37:0/2589217815 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2738059b40 con 0x7f275015aa80 2024-04-26T20:23:28.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:28.593+0000 7f27548da640 1 -- 172.21.15.37:0/2589217815 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone57050", "target": ["mon-mgr", ""]}) -- 0x7f27500d06d0 con 0x7f273006caa0 2024-04-26T20:23:34.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.865+0000 7f2746ffd640 1 -- 172.21.15.37:0/2589217815 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+169 (secure 0 0 0) 0x7f27500d06d0 con 0x7f273006caa0 2024-04-26T20:23:34.866 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:23:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "status": { 2024-04-26T20:23:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "in-progress", 2024-04-26T20:23:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "source": { 2024-04-26T20:23:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "volume": "cephfs", 2024-04-26T20:23:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "subvolume": "subvol19350", 2024-04-26T20:23:34.867 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot": "subvol_snap85010" 2024-04-26T20:23:34.867 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:34.867 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:34.867 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:23:34.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 -- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273006caa0 msgr2=0x7f273006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:34.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273006caa0 0x7f273006ef60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2748032000 tx=0x7f2748009030 comp rx=0 tx=0).stop 2024-04-26T20:23:34.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 -- 172.21.15.37:0/2589217815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275015aa80 msgr2=0x7f27500b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:34.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275015aa80 0x7f27500b3a60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2738007370 tx=0x7f2738007830 comp rx=0 tx=0).stop 2024-04-26T20:23:34.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 -- 172.21.15.37:0/2589217815 shutdown_connections 2024-04-26T20:23:34.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f273006caa0 0x7f273006ef60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:34.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2750150f60 0x7f27501513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:34.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275015aa80 0x7f27500b3a60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:34.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 --2- 172.21.15.37:0/2589217815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c003b70 0x7f275015ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:34.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 -- 172.21.15.37:0/2589217815 >> 172.21.15.37:0/2589217815 conn(0x7f27500d2ff0 msgr2=0x7f275006d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:34.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 -- 172.21.15.37:0/2589217815 shutdown_connections 2024-04-26T20:23:34.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:34.868+0000 7f27548da640 1 -- 172.21.15.37:0/2589217815 wait complete. 2024-04-26T20:23:35.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone57050 2024-04-26T20:23:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.001+0000 7f5d89fe7640 1 Processor -- start 2024-04-26T20:23:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.001+0000 7f5d89fe7640 1 -- start start 2024-04-26T20:23:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d89fe7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5d8415cdf0 0x7f5d840d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d89fe7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5d840d32d0 0x7f5d840d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d89fe7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d840d2590 con 0x7f5d8415d1d0 2024-04-26T20:23:36.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d89fe7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f5d840d4310 con 0x7f5d840d32d0 2024-04-26T20:23:36.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d89fe7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5d840d2870 con 0x7f5d8415cdf0 2024-04-26T20:23:36.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d82ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5d840d32d0 0x7f5d840d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:36.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d83fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5d8415d1d0 0x7f5d840b58b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45444/0 (socket says 172.21.15.37:45444) 2024-04-26T20:23:36.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d83fff640 1 -- 172.21.15.37:0/907782301 learned_addr learned my addr 172.21.15.37:0/907782301 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:36.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d837fe640 1 --2- 172.21.15.37:0/907782301 >> v2:172.21.15.134:3300/0 conn(0x7f5d8415cdf0 0x7f5d840d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:36.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d82ffd640 1 -- 172.21.15.37:0/907782301 >> v2:172.21.15.134:3300/0 conn(0x7f5d8415cdf0 msgr2=0x7f5d840d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:36.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d82ffd640 1 --2- 172.21.15.37:0/907782301 >> v2:172.21.15.134:3300/0 conn(0x7f5d8415cdf0 0x7f5d840d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:36.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d82ffd640 1 -- 172.21.15.37:0/907782301 >> v1:172.21.15.37:6789/0 conn(0x7f5d8415d1d0 legacy=0x7f5d840b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:36.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d82ffd640 1 -- 172.21.15.37:0/907782301 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d840b50b0 con 0x7f5d840d32d0 2024-04-26T20:23:36.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d837fe640 1 --2- 172.21.15.37:0/907782301 >> v2:172.21.15.134:3300/0 conn(0x7f5d8415cdf0 0x7f5d840d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:36.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.002+0000 7f5d827fc640 1 -- 172.21.15.37:0/907782301 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1875602324 0 0) 0x7f5d840d2590 con 0x7f5d8415d1d0 2024-04-26T20:23:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.003+0000 7f5d82ffd640 1 --2- 172.21.15.37:0/907782301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d840d32d0 0x7f5d840d1ad0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5d840d2ce0 tx=0x7f5d7802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d5ed18e62b11d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.003+0000 7f5d827fc640 1 -- 172.21.15.37:0/907782301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78037070 con 0x7f5d840d32d0 2024-04-26T20:23:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.003+0000 7f5d827fc640 1 -- 172.21.15.37:0/907782301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d78032440 con 0x7f5d840d32d0 2024-04-26T20:23:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.003+0000 7f5d827fc640 1 -- 172.21.15.37:0/907782301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78032810 con 0x7f5d840d32d0 2024-04-26T20:23:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.004+0000 7f5d89fe7640 1 -- 172.21.15.37:0/907782301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d840d32d0 msgr2=0x7f5d840d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.004+0000 7f5d89fe7640 1 --2- 172.21.15.37:0/907782301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d840d32d0 0x7f5d840d1ad0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5d840d2ce0 tx=0x7f5d7802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:23:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.004+0000 7f5d89fe7640 1 -- 172.21.15.37:0/907782301 shutdown_connections 2024-04-26T20:23:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.004+0000 7f5d89fe7640 1 --2- 172.21.15.37:0/907782301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d840d32d0 0x7f5d840d1ad0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.004+0000 7f5d89fe7640 1 --2- 172.21.15.37:0/907782301 >> v2:172.21.15.134:3300/0 conn(0x7f5d8415cdf0 0x7f5d840d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.004+0000 7f5d89fe7640 1 -- 172.21.15.37:0/907782301 >> 172.21.15.37:0/907782301 conn(0x7f5d8408ec30 msgr2=0x7f5d8409e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.004+0000 7f5d89fe7640 1 -- 172.21.15.37:0/907782301 shutdown_connections 2024-04-26T20:23:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.004+0000 7f5d89fe7640 1 -- 172.21.15.37:0/907782301 wait complete. 2024-04-26T20:23:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.005+0000 7f5d89fe7640 1 Processor -- start 2024-04-26T20:23:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.005+0000 7f5d89fe7640 1 -- start start 2024-04-26T20:23:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.006+0000 7f5d89fe7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d840d32d0 0x7f5d840cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.006+0000 7f5d89fe7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8415cdf0 0x7f5d840cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.006+0000 7f5d89fe7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8415d1d0 0x7f5d8414eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.006+0000 7f5d89fe7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d840b5fc0 con 0x7f5d8415d1d0 2024-04-26T20:23:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.006+0000 7f5d89fe7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8415fcc0 con 0x7f5d840d32d0 2024-04-26T20:23:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.006+0000 7f5d89fe7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5d840b6140 con 0x7f5d8415cdf0 2024-04-26T20:23:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.006+0000 7f5d83fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8415d1d0 0x7f5d8414eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.006+0000 7f5d83fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8415d1d0 0x7f5d8414eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58796/0 (socket says 172.21.15.37:58796) 2024-04-26T20:23:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.006+0000 7f5d83fff640 1 -- 172.21.15.37:0/589373274 learned_addr learned my addr 172.21.15.37:0/589373274 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.006+0000 7f5d837fe640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d840d32d0 0x7f5d840cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.006+0000 7f5d82ffd640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8415cdf0 0x7f5d840cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.007+0000 7f5d83fff640 1 -- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8415cdf0 msgr2=0x7f5d840cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.007+0000 7f5d83fff640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8415cdf0 0x7f5d840cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.007+0000 7f5d83fff640 1 -- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d840d32d0 msgr2=0x7f5d840cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.007+0000 7f5d83fff640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d840d32d0 0x7f5d840cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.007+0000 7f5d83fff640 1 -- 172.21.15.37:0/589373274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d840b2b30 con 0x7f5d8415d1d0 2024-04-26T20:23:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.007+0000 7f5d83fff640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8415d1d0 0x7f5d8414eca0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00c700 tx=0x7f5d7c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.007+0000 7f5d6bfff640 1 -- 172.21.15.37:0/589373274 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c018070 con 0x7f5d8415d1d0 2024-04-26T20:23:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.007+0000 7f5d6bfff640 1 -- 172.21.15.37:0/589373274 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d7c00ce10 con 0x7f5d8415d1d0 2024-04-26T20:23:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.007+0000 7f5d89fe7640 1 -- 172.21.15.37:0/589373274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d840b2e10 con 0x7f5d8415d1d0 2024-04-26T20:23:36.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.007+0000 7f5d6bfff640 1 -- 172.21.15.37:0/589373274 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00a860 con 0x7f5d8415d1d0 2024-04-26T20:23:36.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.007+0000 7f5d89fe7640 1 -- 172.21.15.37:0/589373274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d840b33d0 con 0x7f5d8415d1d0 2024-04-26T20:23:36.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.008+0000 7f5d89fe7640 1 -- 172.21.15.37:0/589373274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d840b3770 con 0x7f5d8415d1d0 2024-04-26T20:23:36.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.009+0000 7f5d6bfff640 1 -- 172.21.15.37:0/589373274 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f5d7c00aa00 con 0x7f5d8415d1d0 2024-04-26T20:23:36.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.009+0000 7f5d6bfff640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5d7006caa0 0x7f5d7006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:36.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.009+0000 7f5d6bfff640 1 -- 172.21.15.37:0/589373274 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f5d7c08b540 con 0x7f5d8415d1d0 2024-04-26T20:23:36.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.009+0000 7f5d837fe640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5d7006caa0 0x7f5d7006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.010+0000 7f5d837fe640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5d7006caa0 0x7f5d7006ef60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400a990 tx=0x7f5d74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.014+0000 7f5d6bfff640 1 -- 172.21.15.37:0/589373274 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d840b3770 con 0x7f5d8415d1d0 2024-04-26T20:23:36.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:36.237+0000 7f5d89fe7640 1 -- 172.21.15.37:0/589373274 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone57050", "target": ["mon-mgr", ""]}) -- 0x7f5d8406a360 con 0x7f5d7006caa0 2024-04-26T20:23:39.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.867+0000 7f5d6bfff640 1 -- 172.21.15.37:0/589373274 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+272 (secure 0 0 0) 0x7f5d8406a360 con 0x7f5d7006caa0 2024-04-26T20:23:39.868 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: "status": { 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "failed", 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: "source": { 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: "volume": "cephfs", 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: "subvolume": "subvol19350", 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot": "subvol_snap85010" 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: "failure": { 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: "errno": "2", 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: "error_msg": "snapshot 'subvol_snap85010' does not exist" 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:39.870 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:23:39.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 -- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5d7006caa0 msgr2=0x7f5d7006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:39.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5d7006caa0 0x7f5d7006ef60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400a990 tx=0x7f5d74008040 comp rx=0 tx=0).stop 2024-04-26T20:23:39.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 -- 172.21.15.37:0/589373274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8415d1d0 msgr2=0x7f5d8414eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:39.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8415d1d0 0x7f5d8414eca0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00c700 tx=0x7f5d7c00cbc0 comp rx=0 tx=0).stop 2024-04-26T20:23:39.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 -- 172.21.15.37:0/589373274 shutdown_connections 2024-04-26T20:23:39.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8415d1d0 0x7f5d8414eca0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:39.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5d7006caa0 0x7f5d7006ef60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:39.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8415cdf0 0x7f5d840cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:39.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 --2- 172.21.15.37:0/589373274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d840d32d0 0x7f5d840cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:39.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 -- 172.21.15.37:0/589373274 >> 172.21.15.37:0/589373274 conn(0x7f5d8408ec30 msgr2=0x7f5d8415f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:39.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 -- 172.21.15.37:0/589373274 shutdown_connections 2024-04-26T20:23:39.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.870+0000 7f5d89fe7640 1 -- 172.21.15.37:0/589373274 wait complete. 2024-04-26T20:23:39.885 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone57050 2024-04-26T20:23:39.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.994+0000 7f5dc2220640 1 Processor -- start 2024-04-26T20:23:39.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.994+0000 7f5dc2220640 1 -- start start 2024-04-26T20:23:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.995+0000 7f5dc2220640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc0d1990 0x7f5dbc0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.995+0000 7f5dc2220640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dbc074050 con 0x7f5dbc15f3c0 2024-04-26T20:23:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.995+0000 7f5dc2220640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dbc0b1060 con 0x7f5dbc06c950 2024-04-26T20:23:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.995+0000 7f5dc2220640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5dbc0b1340 con 0x7f5dbc0d1990 2024-04-26T20:23:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.995+0000 7f5dbb7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5dbc15f3c0 0x7f5dbc06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45450/0 (socket says 172.21.15.37:45450) 2024-04-26T20:23:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.995+0000 7f5dbb7fe640 1 -- 172.21.15.37:0/1306112027 learned_addr learned my addr 172.21.15.37:0/1306112027 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.995+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 90533984 0 0) 0x7f5dbc074050 con 0x7f5dbc15f3c0 2024-04-26T20:23:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.995+0000 7f5dbbfff640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc0d1990 0x7f5dbc0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.995+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5da4003520 con 0x7f5dbc15f3c0 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2898041971 0 0) 0x7f5da4003520 con 0x7f5dbc15f3c0 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc0d1990 msgr2=0x7f5dbc0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc0d1990 0x7f5dbc0ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 >> v1:172.21.15.134:6789/0 conn(0x7f5dbc06c950 legacy=0x7f5dbc0cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc0b1620 con 0x7f5dbc15f3c0 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f5dac001670 con 0x7f5dbc15f3c0 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 >> v1:172.21.15.37:6789/0 conn(0x7f5dbc15f3c0 legacy=0x7f5dbc06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc0d1990 0x7f5da40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da4003fb0 0x7f5da4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 0x7f5da40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc0b1340 con 0x7f5da4004d40 2024-04-26T20:23:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5da400bbb0 con 0x7f5da4003fb0 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5da400bdb0 con 0x7f5dbc0d1990 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dbbfff640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 0x7f5da40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dbaffd640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da4003fb0 0x7f5da4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dbb7fe640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc0d1990 0x7f5da40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dbbfff640 1 -- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc0d1990 msgr2=0x7f5da40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dbbfff640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc0d1990 0x7f5da40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dbbfff640 1 -- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da4003fb0 msgr2=0x7f5da4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dbbfff640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da4003fb0 0x7f5da4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dbbfff640 1 -- 172.21.15.37:0/1306112027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da4003520 con 0x7f5da4004d40 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dbb7fe640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc0d1990 0x7f5da40039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.996+0000 7f5dbaffd640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da4003fb0 0x7f5da4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.997+0000 7f5dbbfff640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 0x7f5da40051c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009fc0 tx=0x7f5da4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ee36e8eaf68460 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.997+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4009c10 con 0x7f5da4004d40 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.997+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db4036070 con 0x7f5da4004d40 2024-04-26T20:23:40.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.997+0000 7f5dba7fc640 1 -- 172.21.15.37:0/1306112027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4031880 con 0x7f5da4004d40 2024-04-26T20:23:40.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.998+0000 7f5dc2220640 1 -- 172.21.15.37:0/1306112027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 msgr2=0x7f5da40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.998+0000 7f5dc2220640 1 --2- 172.21.15.37:0/1306112027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 0x7f5da40051c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009fc0 tx=0x7f5da4005830 comp rx=0 tx=0).stop 2024-04-26T20:23:40.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.998+0000 7f5dbbfff640 1 -- 172.21.15.37:0/1306112027 reap_dead start 2024-04-26T20:23:40.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.998+0000 7f5dc2220640 1 -- 172.21.15.37:0/1306112027 shutdown_connections 2024-04-26T20:23:40.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.998+0000 7f5dc2220640 1 -- 172.21.15.37:0/1306112027 >> 172.21.15.37:0/1306112027 conn(0x7f5dbc0d1e80 msgr2=0x7f5dbc0d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:40.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.998+0000 7f5dc2220640 1 -- 172.21.15.37:0/1306112027 shutdown_connections 2024-04-26T20:23:40.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.998+0000 7f5dc2220640 1 -- 172.21.15.37:0/1306112027 wait complete. 2024-04-26T20:23:40.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.999+0000 7f5dc2220640 1 Processor -- start 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:39.999+0000 7f5dc2220640 1 -- start start 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dc2220640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 0x7f5dbc14a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dc2220640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc14d0c0 0x7f5dbc14d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dc2220640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dbc1e9a20 0x7f5dbc1e9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dc2220640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc0d1530 con 0x7f5da4004d40 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dc2220640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc0d13b0 con 0x7f5dbc1e9a20 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dc2220640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5dbc15f7a0 con 0x7f5dbc14d0c0 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dbb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 0x7f5dbc14a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dbb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 0x7f5dbc14a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58812/0 (socket says 172.21.15.37:58812) 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dbb7fe640 1 -- 172.21.15.37:0/3163098015 learned_addr learned my addr 172.21.15.37:0/3163098015 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dbb7fe640 1 -- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc14d0c0 msgr2=0x7f5dbc14d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dbb7fe640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc14d0c0 0x7f5dbc14d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dbb7fe640 1 -- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dbc1e9a20 msgr2=0x7f5dbc1e9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dbb7fe640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dbc1e9a20 0x7f5dbc1e9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dbb7fe640 1 -- 172.21.15.37:0/3163098015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc0b4ac0 con 0x7f5da4004d40 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.000+0000 7f5dbb7fe640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 0x7f5dbc14a6c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00e850 tx=0x7f5dac00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.001+0000 7f5da3fff640 1 -- 172.21.15.37:0/3163098015 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac00c850 con 0x7f5da4004d40 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.001+0000 7f5da3fff640 1 -- 172.21.15.37:0/3163098015 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac018070 con 0x7f5da4004d40 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.001+0000 7f5dc2220640 1 -- 172.21.15.37:0/3163098015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc1ea790 con 0x7f5da4004d40 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.001+0000 7f5da3fff640 1 -- 172.21.15.37:0/3163098015 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac00cbf0 con 0x7f5da4004d40 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.001+0000 7f5dc2220640 1 -- 172.21.15.37:0/3163098015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc14b560 con 0x7f5da4004d40 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.002+0000 7f5dc2220640 1 -- 172.21.15.37:0/3163098015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dbc06a360 con 0x7f5da4004d40 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.002+0000 7f5da3fff640 1 -- 172.21.15.37:0/3163098015 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f5dac01d460 con 0x7f5da4004d40 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.002+0000 7f5da3fff640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5da806caa0 0x7f5da806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.003+0000 7f5da3fff640 1 -- 172.21.15.37:0/3163098015 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f5dac08bd40 con 0x7f5da4004d40 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.003+0000 7f5dbaffd640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5da806caa0 0x7f5da806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.003+0000 7f5dbaffd640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5da806caa0 0x7f5da806ef60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc15e0e0 tx=0x7f5db0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:40.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.007+0000 7f5da3fff640 1 -- 172.21.15.37:0/3163098015 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5dbc0b4ac0 con 0x7f5da4004d40 2024-04-26T20:23:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.229+0000 7f5dc2220640 1 -- 172.21.15.37:0/3163098015 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone57050", "target": ["mon-mgr", ""]}) -- 0x7f5dbc0b7e00 con 0x7f5da806caa0 2024-04-26T20:23:40.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.236+0000 7f5da3fff640 1 -- 172.21.15.37:0/3163098015 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+272 (secure 0 0 0) 0x7f5dbc0b7e00 con 0x7f5da806caa0 2024-04-26T20:23:40.237 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:23:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: "status": { 2024-04-26T20:23:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "failed", 2024-04-26T20:23:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "source": { 2024-04-26T20:23:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "volume": "cephfs", 2024-04-26T20:23:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "subvolume": "subvol19350", 2024-04-26T20:23:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot": "subvol_snap85010" 2024-04-26T20:23:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:23:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "failure": { 2024-04-26T20:23:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "errno": "2", 2024-04-26T20:23:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "error_msg": "snapshot 'subvol_snap85010' does not exist" 2024-04-26T20:23:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:23:40.238 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:23:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 -- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5da806caa0 msgr2=0x7f5da806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5da806caa0 0x7f5da806ef60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc15e0e0 tx=0x7f5db0009040 comp rx=0 tx=0).stop 2024-04-26T20:23:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 -- 172.21.15.37:0/3163098015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 msgr2=0x7f5dbc14a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 0x7f5dbc14a6c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00e850 tx=0x7f5dac00ed10 comp rx=0 tx=0).stop 2024-04-26T20:23:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 -- 172.21.15.37:0/3163098015 shutdown_connections 2024-04-26T20:23:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dbc1e9a20 0x7f5dbc1e9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f5da806caa0 0x7f5da806ef60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dbc14d0c0 0x7f5dbc14d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 --2- 172.21.15.37:0/3163098015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4004d40 0x7f5dbc14a6c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 -- 172.21.15.37:0/3163098015 >> 172.21.15.37:0/3163098015 conn(0x7f5dbc0d1e80 msgr2=0x7f5dbc0cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 -- 172.21.15.37:0/3163098015 shutdown_connections 2024-04-26T20:23:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.238+0000 7f5dc2220640 1 -- 172.21.15.37:0/3163098015 wait complete. 2024-04-26T20:23:40.250 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone57050 --force 2024-04-26T20:23:40.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.353+0000 7f98ab2d1640 1 Processor -- start 2024-04-26T20:23:40.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.354+0000 7f98ab2d1640 1 -- start start 2024-04-26T20:23:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.354+0000 7f98ab2d1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f98a415f5d0 0x7f98a40d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.354+0000 7f98ab2d1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f98a40d5640 0x7f98a40b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.354+0000 7f98ab2d1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a40b50f0 con 0x7f98a40d4010 2024-04-26T20:23:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.354+0000 7f98ab2d1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f98a40b5420 con 0x7f98a40d5640 2024-04-26T20:23:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.354+0000 7f98ab2d1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f98a40d0a80 con 0x7f98a415f5d0 2024-04-26T20:23:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.354+0000 7f98a986e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f98a40d5640 0x7f98a40b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.354+0000 7f98a986e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f98a40d5640 0x7f98a40b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58814/0 (socket says 172.21.15.37:58814) 2024-04-26T20:23:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.355+0000 7f98a986e640 1 -- 172.21.15.37:0/2146367486 learned_addr learned my addr 172.21.15.37:0/2146367486 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.355+0000 7f98a906d640 1 --2- 172.21.15.37:0/2146367486 >> v2:172.21.15.134:3300/0 conn(0x7f98a415f5d0 0x7f98a40d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.355+0000 7f98a986e640 1 -- 172.21.15.37:0/2146367486 >> v2:172.21.15.134:3300/0 conn(0x7f98a415f5d0 msgr2=0x7f98a40d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.355+0000 7f98a986e640 1 --2- 172.21.15.37:0/2146367486 >> v2:172.21.15.134:3300/0 conn(0x7f98a415f5d0 0x7f98a40d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.355+0000 7f98a986e640 1 -- 172.21.15.37:0/2146367486 >> v1:172.21.15.134:6789/0 conn(0x7f98a40d4010 legacy=0x7f98a40d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.355+0000 7f98a986e640 1 -- 172.21.15.37:0/2146367486 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a40d0d60 con 0x7f98a40d5640 2024-04-26T20:23:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98a986e640 1 --2- 172.21.15.37:0/2146367486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d5640 0x7f98a40b58b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f98a408f140 tx=0x7f989402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139301bb6261c86e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98a3fff640 1 -- 172.21.15.37:0/2146367486 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894036070 con 0x7f98a40d5640 2024-04-26T20:23:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98a3fff640 1 -- 172.21.15.37:0/2146367486 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9894031600 con 0x7f98a40d5640 2024-04-26T20:23:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98a3fff640 1 -- 172.21.15.37:0/2146367486 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98940319f0 con 0x7f98a40d5640 2024-04-26T20:23:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98ab2d1640 1 -- 172.21.15.37:0/2146367486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d5640 msgr2=0x7f98a40b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98ab2d1640 1 --2- 172.21.15.37:0/2146367486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d5640 0x7f98a40b58b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f98a408f140 tx=0x7f989402a7c0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98ab2d1640 1 -- 172.21.15.37:0/2146367486 shutdown_connections 2024-04-26T20:23:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98ab2d1640 1 --2- 172.21.15.37:0/2146367486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d5640 0x7f98a40b58b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98ab2d1640 1 --2- 172.21.15.37:0/2146367486 >> v2:172.21.15.134:3300/0 conn(0x7f98a415f5d0 0x7f98a40d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98ab2d1640 1 -- 172.21.15.37:0/2146367486 >> 172.21.15.37:0/2146367486 conn(0x7f98a408ebd0 msgr2=0x7f98a409e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98ab2d1640 1 -- 172.21.15.37:0/2146367486 shutdown_connections 2024-04-26T20:23:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.356+0000 7f98ab2d1640 1 -- 172.21.15.37:0/2146367486 wait complete. 2024-04-26T20:23:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.357+0000 7f98ab2d1640 1 Processor -- start 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98ab2d1640 1 -- start start 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98ab2d1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a40d4010 0x7f98a414f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98ab2d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d5640 0x7f98a40b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98ab2d1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a415f5d0 0x7f98a4151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98ab2d1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98a40d34f0 con 0x7f98a40d5640 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98ab2d1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98a415f9b0 con 0x7f98a40d4010 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98ab2d1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98a40b5df0 con 0x7f98a415f5d0 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98a886c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d5640 0x7f98a40b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98a886c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d5640 0x7f98a40b7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58828/0 (socket says 172.21.15.37:58828) 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98a886c640 1 -- 172.21.15.37:0/589143968 learned_addr learned my addr 172.21.15.37:0/589143968 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98a906d640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a40d4010 0x7f98a414f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.358+0000 7f98a986e640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a415f5d0 0x7f98a4151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.359+0000 7f98a886c640 1 -- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a415f5d0 msgr2=0x7f98a4151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.359+0000 7f98a886c640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a415f5d0 0x7f98a4151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.359+0000 7f98a886c640 1 -- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a40d4010 msgr2=0x7f98a414f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.359+0000 7f98a886c640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a40d4010 0x7f98a414f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.359+0000 7f98a886c640 1 -- 172.21.15.37:0/589143968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a414f6f0 con 0x7f98a40d5640 2024-04-26T20:23:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.359+0000 7f98a886c640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d5640 0x7f98a40b7de0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f989800c3c0 tx=0x7f989800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.359+0000 7f98a1ffb640 1 -- 172.21.15.37:0/589143968 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898017070 con 0x7f98a40d5640 2024-04-26T20:23:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.359+0000 7f98a1ffb640 1 -- 172.21.15.37:0/589143968 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98980052f0 con 0x7f98a40d5640 2024-04-26T20:23:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.359+0000 7f98ab2d1640 1 -- 172.21.15.37:0/589143968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a40ba660 con 0x7f98a40d5640 2024-04-26T20:23:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.359+0000 7f98a1ffb640 1 -- 172.21.15.37:0/589143968 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898003850 con 0x7f98a40d5640 2024-04-26T20:23:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.359+0000 7f98ab2d1640 1 -- 172.21.15.37:0/589143968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a40bac20 con 0x7f98a40d5640 2024-04-26T20:23:40.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.360+0000 7f98ab2d1640 1 -- 172.21.15.37:0/589143968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a406a360 con 0x7f98a40d5640 2024-04-26T20:23:40.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.361+0000 7f98a1ffb640 1 -- 172.21.15.37:0/589143968 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f989801a070 con 0x7f98a40d5640 2024-04-26T20:23:40.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.361+0000 7f98a1ffb640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f989006caa0 0x7f989006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.361+0000 7f98a1ffb640 1 -- 172.21.15.37:0/589143968 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f989808b0c0 con 0x7f98a40d5640 2024-04-26T20:23:40.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.361+0000 7f98a906d640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f989006caa0 0x7f989006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.362+0000 7f98a906d640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f989006caa0 0x7f989006ef60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f989c0079a0 tx=0x7f989c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.365+0000 7f98a1ffb640 1 -- 172.21.15.37:0/589143968 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f98a414f6f0 con 0x7f98a40d5640 2024-04-26T20:23:40.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.577+0000 7f98ab2d1640 1 -- 172.21.15.37:0/589143968 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone57050", "force": true, "target": ["mon-mgr", ""]}) -- 0x7f98a40b3e70 con 0x7f989006caa0 2024-04-26T20:23:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.591+0000 7f98a1ffb640 1 -- 172.21.15.37:0/589143968 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f98a40b3e70 con 0x7f989006caa0 2024-04-26T20:23:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 -- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f989006caa0 msgr2=0x7f989006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f989006caa0 0x7f989006ef60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f989c0079a0 tx=0x7f989c008040 comp rx=0 tx=0).stop 2024-04-26T20:23:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 -- 172.21.15.37:0/589143968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d5640 msgr2=0x7f98a40b7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d5640 0x7f98a40b7de0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f989800c3c0 tx=0x7f989800c880 comp rx=0 tx=0).stop 2024-04-26T20:23:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 -- 172.21.15.37:0/589143968 shutdown_connections 2024-04-26T20:23:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f989006caa0 0x7f989006ef60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a415f5d0 0x7f98a4151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d5640 0x7f98a40b7de0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 --2- 172.21.15.37:0/589143968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a40d4010 0x7f98a414f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 -- 172.21.15.37:0/589143968 >> 172.21.15.37:0/589143968 conn(0x7f98a408ebd0 msgr2=0x7f98a40b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 -- 172.21.15.37:0/589143968 shutdown_connections 2024-04-26T20:23:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.593+0000 7f98ab2d1640 1 -- 172.21.15.37:0/589143968 wait complete. 2024-04-26T20:23:40.595 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:40.595+0000 7f9a881f7640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:40.596 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:40.595+0000 7f9a881f7640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:40.596 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:40.595+0000 7f9a881f7640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:40.596 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:40.595+0000 7f9a881f7640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:40.596 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:40.595+0000 7f9a881f7640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:40.605 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol19350 2024-04-26T20:23:40.609 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:40.609+0000 7f9a869f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:40.610 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:40.609+0000 7f9a869f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:40.610 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:40.609+0000 7f9a869f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:40.610 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:40.609+0000 7f9a869f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:40.610 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-26T20:23:40.609+0000 7f9a869f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:23:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.709+0000 7f3187e9b640 1 Processor -- start 2024-04-26T20:23:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.709+0000 7f3187e9b640 1 -- start start 2024-04-26T20:23:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3187e9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f318015d460 0x7f318006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3187e9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3180073c40 0x7f31800ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3187e9b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f318006ece0 con 0x7f31800cd100 2024-04-26T20:23:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3187e9b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f31800add80 con 0x7f3180073c40 2024-04-26T20:23:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3187e9b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f318006efc0 con 0x7f318015d460 2024-04-26T20:23:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3185c37640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f318015d460 0x7f318006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3185436640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3180073c40 0x7f31800ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3185c37640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f318015d460 0x7f318006e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:41914/0 (socket says 172.21.15.37:41914) 2024-04-26T20:23:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3185c37640 1 -- 172.21.15.37:0/1600857585 learned_addr learned my addr 172.21.15.37:0/1600857585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3186438640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f31800cd100 0x7f31800cb3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:33954/0 (socket says 172.21.15.37:33954) 2024-04-26T20:23:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3185436640 1 -- 172.21.15.37:0/1600857585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f318015d460 msgr2=0x7f318006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3185436640 1 --2- 172.21.15.37:0/1600857585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f318015d460 0x7f318006e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3185436640 1 -- 172.21.15.37:0/1600857585 >> v1:172.21.15.134:6789/0 conn(0x7f31800cd100 legacy=0x7f31800cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3185436640 1 -- 172.21.15.37:0/1600857585 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318006f2a0 con 0x7f3180073c40 2024-04-26T20:23:40.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.710+0000 7f3185c37640 1 --2- 172.21.15.37:0/1600857585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f318015d460 0x7f318006e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:40.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.711+0000 7f3185436640 1 --2- 172.21.15.37:0/1600857585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3180073c40 0x7f31800ccbc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31700097a0 tx=0x7f317002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7723e8abd0aa2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.711+0000 7f3184c35640 1 -- 172.21.15.37:0/1600857585 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170037070 con 0x7f3180073c40 2024-04-26T20:23:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.711+0000 7f3184c35640 1 -- 172.21.15.37:0/1600857585 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3170032440 con 0x7f3180073c40 2024-04-26T20:23:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.711+0000 7f3184c35640 1 -- 172.21.15.37:0/1600857585 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170032810 con 0x7f3180073c40 2024-04-26T20:23:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.712+0000 7f3187e9b640 1 -- 172.21.15.37:0/1600857585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3180073c40 msgr2=0x7f31800ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.712+0000 7f3187e9b640 1 --2- 172.21.15.37:0/1600857585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3180073c40 0x7f31800ccbc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31700097a0 tx=0x7f317002a210 comp rx=0 tx=0).stop 2024-04-26T20:23:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.712+0000 7f3187e9b640 1 -- 172.21.15.37:0/1600857585 shutdown_connections 2024-04-26T20:23:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.712+0000 7f3187e9b640 1 --2- 172.21.15.37:0/1600857585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3180073c40 0x7f31800ccbc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.712+0000 7f3187e9b640 1 --2- 172.21.15.37:0/1600857585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f318015d460 0x7f318006e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.712+0000 7f3187e9b640 1 -- 172.21.15.37:0/1600857585 >> 172.21.15.37:0/1600857585 conn(0x7f31800d0d70 msgr2=0x7f31800d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.712+0000 7f3187e9b640 1 -- 172.21.15.37:0/1600857585 shutdown_connections 2024-04-26T20:23:40.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.712+0000 7f3187e9b640 1 -- 172.21.15.37:0/1600857585 wait complete. 2024-04-26T20:23:40.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.713+0000 7f3187e9b640 1 Processor -- start 2024-04-26T20:23:40.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.713+0000 7f3187e9b640 1 -- start start 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3187e9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3180073c40 0x7f31801f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3187e9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31800cd100 0x7f318015ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3187e9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318015d460 0x7f31801f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3187e9b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f318015d9c0 con 0x7f318015d460 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3187e9b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f318015d840 con 0x7f31800cd100 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3187e9b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f31800740a0 con 0x7f3180073c40 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3186438640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318015d460 0x7f31801f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3186438640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318015d460 0x7f31801f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58834/0 (socket says 172.21.15.37:58834) 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3186438640 1 -- 172.21.15.37:0/4226694412 learned_addr learned my addr 172.21.15.37:0/4226694412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3185436640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31800cd100 0x7f318015ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3185c37640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3180073c40 0x7f31801f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3186438640 1 -- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3180073c40 msgr2=0x7f31801f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3186438640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3180073c40 0x7f31801f85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3186438640 1 -- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31800cd100 msgr2=0x7f318015ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3186438640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31800cd100 0x7f318015ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3186438640 1 -- 172.21.15.37:0/4226694412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31801f5000 con 0x7f318015d460 2024-04-26T20:23:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.714+0000 7f3186438640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318015d460 0x7f31801f57a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f317800c780 tx=0x7f317800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.715+0000 7f31767fc640 1 -- 172.21.15.37:0/4226694412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178017070 con 0x7f318015d460 2024-04-26T20:23:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.715+0000 7f31767fc640 1 -- 172.21.15.37:0/4226694412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317800a640 con 0x7f318015d460 2024-04-26T20:23:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.715+0000 7f3187e9b640 1 -- 172.21.15.37:0/4226694412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31801f8b30 con 0x7f318015d460 2024-04-26T20:23:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.715+0000 7f31767fc640 1 -- 172.21.15.37:0/4226694412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317800aa30 con 0x7f318015d460 2024-04-26T20:23:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.715+0000 7f3187e9b640 1 -- 172.21.15.37:0/4226694412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31801f90b0 con 0x7f318015d460 2024-04-26T20:23:40.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.716+0000 7f3187e9b640 1 -- 172.21.15.37:0/4226694412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318006a360 con 0x7f318015d460 2024-04-26T20:23:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.716+0000 7f31767fc640 1 -- 172.21.15.37:0/4226694412 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f317801a070 con 0x7f318015d460 2024-04-26T20:23:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.717+0000 7f31767fc640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f316406caa0 0x7f316406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.717+0000 7f31767fc640 1 -- 172.21.15.37:0/4226694412 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f317808b310 con 0x7f318015d460 2024-04-26T20:23:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.717+0000 7f3185c37640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f316406caa0 0x7f316406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.718+0000 7f3185c37640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f316406caa0 0x7f316406ef60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f317c009630 tx=0x7f317c009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:40.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.721+0000 7f31767fc640 1 -- 172.21.15.37:0/4226694412 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f31801f5000 con 0x7f318015d460 2024-04-26T20:23:40.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:40.935+0000 7f3187e9b640 1 -- 172.21.15.37:0/4226694412 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol19350", "target": ["mon-mgr", ""]}) -- 0x7f31800c79a0 con 0x7f316406caa0 2024-04-26T20:23:42.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:42.021+0000 7f31767fc640 1 -- 172.21.15.37:0/4226694412 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f3178055570 con 0x7f318015d460 2024-04-26T20:23:44.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.880+0000 7f31767fc640 1 -- 172.21.15.37:0/4226694412 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f31800c79a0 con 0x7f316406caa0 2024-04-26T20:23:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.883+0000 7f3187e9b640 1 -- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f316406caa0 msgr2=0x7f316406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.883+0000 7f3187e9b640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f316406caa0 0x7f316406ef60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f317c009630 tx=0x7f317c009240 comp rx=0 tx=0).stop 2024-04-26T20:23:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.883+0000 7f3187e9b640 1 -- 172.21.15.37:0/4226694412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318015d460 msgr2=0x7f31801f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.883+0000 7f3187e9b640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318015d460 0x7f31801f57a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f317800c780 tx=0x7f317800cc40 comp rx=0 tx=0).stop 2024-04-26T20:23:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.883+0000 7f3187e9b640 1 -- 172.21.15.37:0/4226694412 shutdown_connections 2024-04-26T20:23:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.883+0000 7f3187e9b640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318015d460 0x7f31801f57a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.883+0000 7f3187e9b640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31800cd100 0x7f318015ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.883+0000 7f3187e9b640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f316406caa0 0x7f316406ef60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.883+0000 7f3187e9b640 1 --2- 172.21.15.37:0/4226694412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3180073c40 0x7f31801f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.883+0000 7f3187e9b640 1 -- 172.21.15.37:0/4226694412 >> 172.21.15.37:0/4226694412 conn(0x7f31800d0d70 msgr2=0x7f318006dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.884+0000 7f3187e9b640 1 -- 172.21.15.37:0/4226694412 shutdown_connections 2024-04-26T20:23:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44.884+0000 7f3187e9b640 1 -- 172.21.15.37:0/4226694412 wait complete. 2024-04-26T20:23:44.901 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-26T20:23:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:23:44.957 INFO:teuthology.orchestra.run.smithi037.stdout:3 2024-04-26T20:23:44.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:23:49.652 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:23:49.655 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:23:49.965 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-26T20:23:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:23:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:2 2024-04-26T20:23:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:23:50.034 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:23:50.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:23:50.075 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:23:50.125 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:23:50.125 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:23:50.125 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:50.125 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:50.142 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:50.142 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:50.199 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:23:50.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.341+0000 7fe31d3f0640 1 Processor -- start 2024-04-26T20:23:50.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.341+0000 7fe31d3f0640 1 -- start start 2024-04-26T20:23:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.341+0000 7fe31d3f0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe31806f790 0x7fe318074030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe31d3f0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe318161b80 con 0x7fe3180b6b10 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe31d3f0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3180b4400 con 0x7fe318160d70 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe31d3f0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe318074570 con 0x7fe31806f790 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe3177fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe3180b6b10 0x7fe31815de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53824/0 (socket says 172.21.15.37:53824) 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe3177fe640 1 -- 172.21.15.37:0/3493111658 learned_addr learned my addr 172.21.15.37:0/3493111658 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2812954622 0 0) 0x7fe318161b80 con 0x7fe3180b6b10 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe300003520 con 0x7fe3180b6b10 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe316ffd640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe31806f790 0x7fe318074030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 863550664 0 0) 0x7fe300003520 con 0x7fe3180b6b10 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe31806f790 msgr2=0x7fe318074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe31806f790 0x7fe318074030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 >> v1:172.21.15.134:6789/0 conn(0x7fe318160d70 legacy=0x7fe31815d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3180b4120 con 0x7fe3180b6b10 2024-04-26T20:23:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fe3100019f0 con 0x7fe3180b6b10 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 >> v1:172.21.15.37:6789/0 conn(0x7fe3180b6b10 legacy=0x7fe31815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3000039e0 0x7fe300003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe300004800 0x7fe300004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe31806f790 0x7fe300005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe318074570 con 0x7fe3000039e0 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe30000bb50 con 0x7fe300004800 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe30000bd50 con 0x7fe31806f790 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.342+0000 7fe316ffd640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3000039e0 0x7fe300003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.343+0000 7fe3177fe640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe31806f790 0x7fe300005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.343+0000 7fe3167fc640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe300004800 0x7fe300004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.343+0000 7fe316ffd640 1 -- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe31806f790 msgr2=0x7fe300005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.343+0000 7fe316ffd640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe31806f790 0x7fe300005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.343+0000 7fe316ffd640 1 -- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe300004800 msgr2=0x7fe300004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:50.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.343+0000 7fe316ffd640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe300004800 0x7fe300004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.343+0000 7fe316ffd640 1 -- 172.21.15.37:0/3493111658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe300003520 con 0x7fe3000039e0 2024-04-26T20:23:50.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.343+0000 7fe3167fc640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe300004800 0x7fe300004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:50.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.343+0000 7fe316ffd640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3000039e0 0x7fe300003e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe30800ba30 tx=0x7fe300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00e8edd7753b85f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:50.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.344+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308030070 con 0x7fe3000039e0 2024-04-26T20:23:50.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.344+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30802ade0 con 0x7fe3000039e0 2024-04-26T20:23:50.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.344+0000 7fe315ffb640 1 -- 172.21.15.37:0/3493111658 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30800b600 con 0x7fe3000039e0 2024-04-26T20:23:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.344+0000 7fe31d3f0640 1 -- 172.21.15.37:0/3493111658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3000039e0 msgr2=0x7fe300003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.344+0000 7fe31d3f0640 1 --2- 172.21.15.37:0/3493111658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3000039e0 0x7fe300003e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe30800ba30 tx=0x7fe300005830 comp rx=0 tx=0).stop 2024-04-26T20:23:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.344+0000 7fe3177fe640 1 -- 172.21.15.37:0/3493111658 reap_dead start 2024-04-26T20:23:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.344+0000 7fe31d3f0640 1 -- 172.21.15.37:0/3493111658 shutdown_connections 2024-04-26T20:23:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.344+0000 7fe31d3f0640 1 -- 172.21.15.37:0/3493111658 >> 172.21.15.37:0/3493111658 conn(0x7fe3180d0d50 msgr2=0x7fe3180d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.344+0000 7fe31d3f0640 1 -- 172.21.15.37:0/3493111658 shutdown_connections 2024-04-26T20:23:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.344+0000 7fe31d3f0640 1 -- 172.21.15.37:0/3493111658 wait complete. 2024-04-26T20:23:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.345+0000 7fe31d3f0640 1 Processor -- start 2024-04-26T20:23:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.345+0000 7fe31d3f0640 1 -- start start 2024-04-26T20:23:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.346+0000 7fe31d3f0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3000039e0 0x7fe318136b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.346+0000 7fe31d3f0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe31815ca60 0x7fe3180abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.346+0000 7fe31d3f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe318145420 0x7fe3181458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.346+0000 7fe31d3f0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3180735b0 con 0x7fe318145420 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.346+0000 7fe31d3f0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3180732d0 con 0x7fe31815ca60 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.346+0000 7fe31d3f0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe31815e5a0 con 0x7fe3000039e0 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.346+0000 7fe3177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe318145420 0x7fe3181458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.346+0000 7fe3177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe318145420 0x7fe3181458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37064/0 (socket says 172.21.15.37:37064) 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.346+0000 7fe3177fe640 1 -- 172.21.15.37:0/1538954567 learned_addr learned my addr 172.21.15.37:0/1538954567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.346+0000 7fe3167fc640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe31815ca60 0x7fe3180abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe3177fe640 1 -- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3000039e0 msgr2=0x7fe318136b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe316ffd640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3000039e0 0x7fe318136b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe3177fe640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3000039e0 0x7fe318136b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe3177fe640 1 -- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe31815ca60 msgr2=0x7fe3180abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe3177fe640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe31815ca60 0x7fe3180abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe3177fe640 1 -- 172.21.15.37:0/1538954567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3181370d0 con 0x7fe318145420 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe316ffd640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3000039e0 0x7fe318136b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe3177fe640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe318145420 0x7fe3181458d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe318076150 tx=0x7fe31000a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe2ff7fe640 1 -- 172.21.15.37:0/1538954567 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31000ac40 con 0x7fe318145420 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe2ff7fe640 1 -- 172.21.15.37:0/1538954567 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe310019070 con 0x7fe318145420 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe2ff7fe640 1 -- 172.21.15.37:0/1538954567 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3100133f0 con 0x7fe318145420 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe31d3f0640 1 -- 172.21.15.37:0/1538954567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31815b9c0 con 0x7fe318145420 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.347+0000 7fe31d3f0640 1 -- 172.21.15.37:0/1538954567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31815bf80 con 0x7fe318145420 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.348+0000 7fe31d3f0640 1 -- 172.21.15.37:0/1538954567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31806a360 con 0x7fe318145420 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.348+0000 7fe2ff7fe640 1 -- 172.21.15.37:0/1538954567 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fe310013590 con 0x7fe318145420 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.349+0000 7fe2ff7fe640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe30406cbc0 0x7fe30406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.349+0000 7fe2ff7fe640 1 -- 172.21.15.37:0/1538954567 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7fe31008c6a0 con 0x7fe318145420 2024-04-26T20:23:50.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.349+0000 7fe316ffd640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe30406cbc0 0x7fe30406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.349+0000 7fe316ffd640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe30406cbc0 0x7fe30406f080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe308035000 tx=0x7fe308030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:50.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.353+0000 7fe2ff7fe640 1 -- 172.21.15.37:0/1538954567 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe31005a760 con 0x7fe318145420 2024-04-26T20:23:50.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.538+0000 7fe31d3f0640 1 -- 172.21.15.37:0/1538954567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe31806ff40 con 0x7fe318145420 2024-04-26T20:23:50.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.539+0000 7fe2ff7fe640 1 -- 172.21.15.37:0/1538954567 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16746 (secure 0 0 0) 0x7fe31005a060 con 0x7fe318145420 2024-04-26T20:23:50.541 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:23:50.541 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:23:34.988080+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:22:39.762703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:22:41.066360+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:23:50.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 -- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe30406cbc0 msgr2=0x7fe30406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe30406cbc0 0x7fe30406f080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe308035000 tx=0x7fe308030000 comp rx=0 tx=0).stop 2024-04-26T20:23:50.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 -- 172.21.15.37:0/1538954567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe318145420 msgr2=0x7fe3181458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe318145420 0x7fe3181458d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe318076150 tx=0x7fe31000a640 comp rx=0 tx=0).stop 2024-04-26T20:23:50.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 -- 172.21.15.37:0/1538954567 shutdown_connections 2024-04-26T20:23:50.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe318145420 0x7fe3181458d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe31815ca60 0x7fe3180abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe30406cbc0 0x7fe30406f080 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 --2- 172.21.15.37:0/1538954567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3000039e0 0x7fe318136b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 -- 172.21.15.37:0/1538954567 >> 172.21.15.37:0/1538954567 conn(0x7fe3180d0d50 msgr2=0x7fe31815e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:50.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 -- 172.21.15.37:0/1538954567 shutdown_connections 2024-04-26T20:23:50.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.541+0000 7fe31d3f0640 1 -- 172.21.15.37:0/1538954567 wait complete. 2024-04-26T20:23:50.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:22:39.762703+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:22:41.066360+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:23:50.554 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:23:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.657+0000 7ffa23293640 1 Processor -- start 2024-04-26T20:23:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.657+0000 7ffa23293640 1 -- start start 2024-04-26T20:23:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.658+0000 7ffa23293640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffa1c15f470 0x7ffa1c0d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.658+0000 7ffa23293640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 0x7ffa1c0d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.658+0000 7ffa23293640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1c0d2ed0 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.658+0000 7ffa23293640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1c0d1190 con 0x7ffa1c15f470 2024-04-26T20:23:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.658+0000 7ffa23293640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c0d31b0 con 0x7ffa1c0d0d30 2024-04-26T20:23:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.658+0000 7ffa2102f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffa1c15f470 0x7ffa1c0d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.658+0000 7ffa2082e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 0x7ffa1c0d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.658+0000 7ffa21830640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ffa1c0d1fe0 0x7ffa1c162fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:60482/0 (socket says 172.21.15.37:60482) 2024-04-26T20:23:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.658+0000 7ffa21830640 1 -- 172.21.15.37:0/1534911651 learned_addr learned my addr 172.21.15.37:0/1534911651 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.658+0000 7ffa2102f640 1 --2- 172.21.15.37:0/1534911651 >> v2:172.21.15.134:3300/0 conn(0x7ffa1c15f470 0x7ffa1c0d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:52112/0 (socket says 172.21.15.37:52112) 2024-04-26T20:23:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.658+0000 7ffa2082e640 1 --2- 172.21.15.37:0/1534911651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 0x7ffa1c0d1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:37278/0 (socket says 172.21.15.37:37278) 2024-04-26T20:23:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.659+0000 7ffa2102f640 1 -- 172.21.15.37:0/1534911651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 msgr2=0x7ffa1c0d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.659+0000 7ffa2102f640 1 --2- 172.21.15.37:0/1534911651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 0x7ffa1c0d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.659+0000 7ffa2102f640 1 -- 172.21.15.37:0/1534911651 >> v1:172.21.15.134:6789/0 conn(0x7ffa1c0d1fe0 legacy=0x7ffa1c162fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.659+0000 7ffa2102f640 1 -- 172.21.15.37:0/1534911651 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c0d3490 con 0x7ffa1c15f470 2024-04-26T20:23:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.659+0000 7ffa2082e640 1 --2- 172.21.15.37:0/1534911651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 0x7ffa1c0d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.659+0000 7ffa2102f640 1 --2- 172.21.15.37:0/1534911651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c15f470 0x7ffa1c0d07f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffa180097d0 tx=0x7ffa1802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=821461c707ef697f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.660+0000 7ffa13fff640 1 -- 172.21.15.37:0/1534911651 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18037070 con 0x7ffa1c15f470 2024-04-26T20:23:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.660+0000 7ffa13fff640 1 -- 172.21.15.37:0/1534911651 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa18032440 con 0x7ffa1c15f470 2024-04-26T20:23:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.660+0000 7ffa13fff640 1 -- 172.21.15.37:0/1534911651 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18032810 con 0x7ffa1c15f470 2024-04-26T20:23:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.660+0000 7ffa23293640 1 -- 172.21.15.37:0/1534911651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c15f470 msgr2=0x7ffa1c0d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.660+0000 7ffa23293640 1 --2- 172.21.15.37:0/1534911651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c15f470 0x7ffa1c0d07f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffa180097d0 tx=0x7ffa1802a1c0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.660+0000 7ffa23293640 1 -- 172.21.15.37:0/1534911651 shutdown_connections 2024-04-26T20:23:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.660+0000 7ffa23293640 1 --2- 172.21.15.37:0/1534911651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 0x7ffa1c0d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.660+0000 7ffa23293640 1 --2- 172.21.15.37:0/1534911651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c15f470 0x7ffa1c0d07f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.660+0000 7ffa23293640 1 -- 172.21.15.37:0/1534911651 >> 172.21.15.37:0/1534911651 conn(0x7ffa1c09e510 msgr2=0x7ffa1c0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.660+0000 7ffa23293640 1 -- 172.21.15.37:0/1534911651 shutdown_connections 2024-04-26T20:23:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.660+0000 7ffa23293640 1 -- 172.21.15.37:0/1534911651 wait complete. 2024-04-26T20:23:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.661+0000 7ffa23293640 1 Processor -- start 2024-04-26T20:23:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.661+0000 7ffa23293640 1 -- start start 2024-04-26T20:23:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.662+0000 7ffa23293640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 0x7ffa1c0b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.662+0000 7ffa23293640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa1c0d1fe0 0x7ffa1c1f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.662+0000 7ffa23293640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c15f470 0x7ffa1c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.662+0000 7ffa23293640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c15f850 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.662+0000 7ffa23293640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c0d3a50 con 0x7ffa1c15f470 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.662+0000 7ffa23293640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c0d3f70 con 0x7ffa1c0d0d30 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.662+0000 7ffa2082e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa1c0d1fe0 0x7ffa1c1f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.662+0000 7ffa2082e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa1c0d1fe0 0x7ffa1c1f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37080/0 (socket says 172.21.15.37:37080) 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.662+0000 7ffa2082e640 1 -- 172.21.15.37:0/1802082540 learned_addr learned my addr 172.21.15.37:0/1802082540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.662+0000 7ffa21830640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c15f470 0x7ffa1c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.662+0000 7ffa2102f640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 0x7ffa1c0b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.663+0000 7ffa2082e640 1 -- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 msgr2=0x7ffa1c0b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.663+0000 7ffa2082e640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 0x7ffa1c0b7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.663+0000 7ffa2082e640 1 -- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c15f470 msgr2=0x7ffa1c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.663+0000 7ffa2082e640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c15f470 0x7ffa1c0cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.663+0000 7ffa2082e640 1 -- 172.21.15.37:0/1802082540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c0b8230 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.663+0000 7ffa2082e640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa1c0d1fe0 0x7ffa1c1f6170 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00e960 tx=0x7ffa0c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.663+0000 7ffa11ffb640 1 -- 172.21.15.37:0/1802082540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c017070 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.663+0000 7ffa11ffb640 1 -- 172.21.15.37:0/1802082540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0c00ca00 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.663+0000 7ffa23293640 1 -- 172.21.15.37:0/1802082540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1c1f66b0 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.663+0000 7ffa11ffb640 1 -- 172.21.15.37:0/1802082540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c00cdd0 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.663+0000 7ffa23293640 1 -- 172.21.15.37:0/1802082540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1c0bab30 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.664+0000 7ffa23293640 1 -- 172.21.15.37:0/1802082540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa1c06a360 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.665+0000 7ffa11ffb640 1 -- 172.21.15.37:0/1802082540 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7ffa0c01a070 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.665+0000 7ffa11ffb640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa0006cbc0 0x7ffa0006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.665+0000 7ffa11ffb640 1 -- 172.21.15.37:0/1802082540 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7ffa0c08be20 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.666+0000 7ffa2102f640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa0006cbc0 0x7ffa0006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.666+0000 7ffa2102f640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa0006cbc0 0x7ffa0006f080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa18034000 tx=0x7ffa1802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:50.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.670+0000 7ffa11ffb640 1 -- 172.21.15.37:0/1802082540 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa0c059850 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.858+0000 7ffa23293640 1 -- 172.21.15.37:0/1802082540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffa1c0d5870 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.858+0000 7ffa11ffb640 1 -- 172.21.15.37:0/1802082540 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7ffa0c05db50 con 0x7ffa1c0d1fe0 2024-04-26T20:23:50.860 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:23:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 -- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa0006cbc0 msgr2=0x7ffa0006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa0006cbc0 0x7ffa0006f080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa18034000 tx=0x7ffa1802e000 comp rx=0 tx=0).stop 2024-04-26T20:23:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 -- 172.21.15.37:0/1802082540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa1c0d1fe0 msgr2=0x7ffa1c1f6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa1c0d1fe0 0x7ffa1c1f6170 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00e960 tx=0x7ffa0c00ee20 comp rx=0 tx=0).stop 2024-04-26T20:23:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 -- 172.21.15.37:0/1802082540 shutdown_connections 2024-04-26T20:23:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c15f470 0x7ffa1c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa1c0d1fe0 0x7ffa1c1f6170 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa0006cbc0 0x7ffa0006f080 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 --2- 172.21.15.37:0/1802082540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa1c0d0d30 0x7ffa1c0b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 -- 172.21.15.37:0/1802082540 >> 172.21.15.37:0/1802082540 conn(0x7ffa1c09e510 msgr2=0x7ffa1c0b5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:50.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 -- 172.21.15.37:0/1802082540 shutdown_connections 2024-04-26T20:23:50.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.861+0000 7ffa23293640 1 -- 172.21.15.37:0/1802082540 wait complete. 2024-04-26T20:23:50.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:23:50.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.975+0000 7f4d8cb9d640 1 Processor -- start 2024-04-26T20:23:50.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.975+0000 7f4d8cb9d640 1 -- start start 2024-04-26T20:23:50.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.975+0000 7f4d8cb9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4d8815f9c0 0x7f4d880d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.975+0000 7f4d8cb9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d880cf990 0x7f4d880cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.975+0000 7f4d8cb9d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d880b2550 con 0x7f4d88163380 2024-04-26T20:23:50.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.975+0000 7f4d8cb9d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4d880d0310 con 0x7f4d8815f9c0 2024-04-26T20:23:50.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.975+0000 7f4d8cb9d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d880b3760 con 0x7f4d880cf990 2024-04-26T20:23:50.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.976+0000 7f4d8659c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4d8815f9c0 0x7f4d880d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.976+0000 7f4d7dd9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d880cf990 0x7f4d880cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.976+0000 7f4d8659c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4d8815f9c0 0x7f4d880d1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:52124/0 (socket says 172.21.15.37:52124) 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.976+0000 7f4d7dd9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d880cf990 0x7f4d880cfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:37290/0 (socket says 172.21.15.37:37290) 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.976+0000 7f4d8659c640 1 -- 172.21.15.37:0/219754731 learned_addr learned my addr 172.21.15.37:0/219754731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.976+0000 7f4d8659c640 1 -- 172.21.15.37:0/219754731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d880cf990 msgr2=0x7f4d880cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.976+0000 7f4d8659c640 1 --2- 172.21.15.37:0/219754731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d880cf990 0x7f4d880cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.976+0000 7f4d8659c640 1 -- 172.21.15.37:0/219754731 >> v1:172.21.15.134:6789/0 conn(0x7f4d88163380 legacy=0x7f4d88163810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.976+0000 7f4d8659c640 1 -- 172.21.15.37:0/219754731 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8806dcf0 con 0x7f4d8815f9c0 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.976+0000 7f4d7dd9b640 1 --2- 172.21.15.37:0/219754731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d880cf990 0x7f4d880cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.976+0000 7f4d85d9b640 1 -- 172.21.15.37:0/219754731 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1677474539 0 0) 0x7f4d80001490 con 0x7f4d88163380 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.977+0000 7f4d8659c640 1 --2- 172.21.15.37:0/219754731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d8815f9c0 0x7f4d880d1860 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009870 tx=0x7f4d7802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec33019e609657bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:50.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.977+0000 7f4d85d9b640 1 -- 172.21.15.37:0/219754731 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78037070 con 0x7f4d8815f9c0 2024-04-26T20:23:50.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.977+0000 7f4d85d9b640 1 -- 172.21.15.37:0/219754731 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d78032440 con 0x7f4d8815f9c0 2024-04-26T20:23:50.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.977+0000 7f4d85d9b640 1 -- 172.21.15.37:0/219754731 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78032810 con 0x7f4d8815f9c0 2024-04-26T20:23:50.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.977+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/219754731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d8815f9c0 msgr2=0x7f4d880d1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.978+0000 7f4d8cb9d640 1 --2- 172.21.15.37:0/219754731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d8815f9c0 0x7f4d880d1860 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009870 tx=0x7f4d7802a1c0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.978+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/219754731 shutdown_connections 2024-04-26T20:23:50.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.978+0000 7f4d8cb9d640 1 --2- 172.21.15.37:0/219754731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d880cf990 0x7f4d880cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.978+0000 7f4d8cb9d640 1 --2- 172.21.15.37:0/219754731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d8815f9c0 0x7f4d880d1860 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.978+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/219754731 >> 172.21.15.37:0/219754731 conn(0x7f4d880d2f80 msgr2=0x7f4d880d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:50.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.978+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/219754731 shutdown_connections 2024-04-26T20:23:50.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.978+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/219754731 wait complete. 2024-04-26T20:23:50.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.979+0000 7f4d8cb9d640 1 Processor -- start 2024-04-26T20:23:50.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.979+0000 7f4d8cb9d640 1 -- start start 2024-04-26T20:23:50.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d8cb9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d880cf990 0x7f4d88160df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d8cb9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8815f9c0 0x7f4d880c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d8cb9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d88163380 0x7f4d880c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d8cb9d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8806e590 con 0x7f4d8815f9c0 2024-04-26T20:23:50.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d8cb9d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8806e2b0 con 0x7f4d880cf990 2024-04-26T20:23:50.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d8cb9d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8806e870 con 0x7f4d88163380 2024-04-26T20:23:50.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d7dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8815f9c0 0x7f4d880c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d7dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8815f9c0 0x7f4d880c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37082/0 (socket says 172.21.15.37:37082) 2024-04-26T20:23:50.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d7dd9b640 1 -- 172.21.15.37:0/1070023826 learned_addr learned my addr 172.21.15.37:0/1070023826 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:50.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d8659c640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d880cf990 0x7f4d88160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d86d9d640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d88163380 0x7f4d880c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d7dd9b640 1 -- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d88163380 msgr2=0x7f4d880c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d7dd9b640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d88163380 0x7f4d880c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d7dd9b640 1 -- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d880cf990 msgr2=0x7f4d88160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:50.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d7dd9b640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d880cf990 0x7f4d88160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.980+0000 7f4d7dd9b640 1 -- 172.21.15.37:0/1070023826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d88161790 con 0x7f4d8815f9c0 2024-04-26T20:23:50.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.981+0000 7f4d7dd9b640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8815f9c0 0x7f4d880c8840 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000e990 tx=0x7f4d7000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:50.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.981+0000 7f4d7f7fe640 1 -- 172.21.15.37:0/1070023826 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70017070 con 0x7f4d8815f9c0 2024-04-26T20:23:50.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.981+0000 7f4d7f7fe640 1 -- 172.21.15.37:0/1070023826 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7000c9e0 con 0x7f4d8815f9c0 2024-04-26T20:23:50.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.981+0000 7f4d7f7fe640 1 -- 172.21.15.37:0/1070023826 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000cdb0 con 0x7f4d8815f9c0 2024-04-26T20:23:50.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.981+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/1070023826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d88161a70 con 0x7f4d8815f9c0 2024-04-26T20:23:50.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.981+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/1070023826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d88162030 con 0x7f4d8815f9c0 2024-04-26T20:23:50.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.982+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/1070023826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d8806a360 con 0x7f4d8815f9c0 2024-04-26T20:23:50.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.982+0000 7f4d7f7fe640 1 -- 172.21.15.37:0/1070023826 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f4d7001a070 con 0x7f4d8815f9c0 2024-04-26T20:23:50.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.983+0000 7f4d7f7fe640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4d6806cbc0 0x7f4d6806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:50.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.983+0000 7f4d7f7fe640 1 -- 172.21.15.37:0/1070023826 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f4d7008bd90 con 0x7f4d8815f9c0 2024-04-26T20:23:50.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.983+0000 7f4d8659c640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4d6806cbc0 0x7f4d6806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.983+0000 7f4d8659c640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4d6806cbc0 0x7f4d6806f080 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d78034000 tx=0x7f4d7802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:50.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:50.987+0000 7f4d7f7fe640 1 -- 172.21.15.37:0/1070023826 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d7008f030 con 0x7f4d8815f9c0 2024-04-26T20:23:51.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.171+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/1070023826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4d880b2000 con 0x7f4d8815f9c0 2024-04-26T20:23:51.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.171+0000 7f4d7f7fe640 1 -- 172.21.15.37:0/1070023826 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f4d70059890 con 0x7f4d8815f9c0 2024-04-26T20:23:51.173 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:23:51.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4d6806cbc0 msgr2=0x7f4d6806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4d6806cbc0 0x7f4d6806f080 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d78034000 tx=0x7f4d7802e000 comp rx=0 tx=0).stop 2024-04-26T20:23:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/1070023826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8815f9c0 msgr2=0x7f4d880c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8815f9c0 0x7f4d880c8840 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000e990 tx=0x7f4d7000ee50 comp rx=0 tx=0).stop 2024-04-26T20:23:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/1070023826 shutdown_connections 2024-04-26T20:23:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4d6806cbc0 0x7f4d6806f080 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d88163380 0x7f4d880c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8815f9c0 0x7f4d880c8840 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 --2- 172.21.15.37:0/1070023826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d880cf990 0x7f4d88160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/1070023826 >> 172.21.15.37:0/1070023826 conn(0x7f4d880d2f80 msgr2=0x7f4d8815a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/1070023826 shutdown_connections 2024-04-26T20:23:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.173+0000 7f4d8cb9d640 1 -- 172.21.15.37:0/1070023826 wait complete. 2024-04-26T20:23:51.186 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:23:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.292+0000 7fa50e015640 1 Processor -- start 2024-04-26T20:23:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.292+0000 7fa50e015640 1 -- start start 2024-04-26T20:23:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa50e015640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa5080b2290 0x7fa5080b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa50e015640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa5080d26f0 0x7fa5080cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa50e015640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5080cd640 con 0x7fa5080cf870 2024-04-26T20:23:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa50e015640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa5080d2b20 con 0x7fa5080d26f0 2024-04-26T20:23:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa50e015640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa5080cd920 con 0x7fa5080b2290 2024-04-26T20:23:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa507fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa5080d26f0 0x7fa5080cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa506ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa5080cf870 0x7fa5080d1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53826/0 (socket says 172.21.15.37:53826) 2024-04-26T20:23:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa507fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa5080d26f0 0x7fa5080cfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37084/0 (socket says 172.21.15.37:37084) 2024-04-26T20:23:51.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa506ffd640 1 -- 172.21.15.37:0/929340306 learned_addr learned my addr 172.21.15.37:0/929340306 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:51.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa507fff640 1 -- 172.21.15.37:0/929340306 >> v2:172.21.15.134:3300/0 conn(0x7fa5080b2290 msgr2=0x7fa5080b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:51.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa5077fe640 1 --2- 172.21.15.37:0/929340306 >> v2:172.21.15.134:3300/0 conn(0x7fa5080b2290 0x7fa5080b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.293+0000 7fa507fff640 1 --2- 172.21.15.37:0/929340306 >> v2:172.21.15.134:3300/0 conn(0x7fa5080b2290 0x7fa5080b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.294+0000 7fa507fff640 1 -- 172.21.15.37:0/929340306 >> v1:172.21.15.37:6789/0 conn(0x7fa5080cf870 legacy=0x7fa5080d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.294+0000 7fa507fff640 1 -- 172.21.15.37:0/929340306 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5080cdc00 con 0x7fa5080d26f0 2024-04-26T20:23:51.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.294+0000 7fa5067fc640 1 -- 172.21.15.37:0/929340306 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2527958465 0 0) 0x7fa5080cd640 con 0x7fa5080cf870 2024-04-26T20:23:51.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.294+0000 7fa5077fe640 1 --2- 172.21.15.37:0/929340306 >> v2:172.21.15.134:3300/0 conn(0x7fa5080b2290 0x7fa5080b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:51.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.294+0000 7fa507fff640 1 --2- 172.21.15.37:0/929340306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5080d26f0 0x7fa5080cfd00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa508073710 tx=0x7fa50002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe7cb940e30f66d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:51.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.294+0000 7fa5067fc640 1 -- 172.21.15.37:0/929340306 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500007740 con 0x7fa5080d26f0 2024-04-26T20:23:51.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.294+0000 7fa5067fc640 1 -- 172.21.15.37:0/929340306 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5000078e0 con 0x7fa5080d26f0 2024-04-26T20:23:51.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.294+0000 7fa5067fc640 1 -- 172.21.15.37:0/929340306 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500007cb0 con 0x7fa5080d26f0 2024-04-26T20:23:51.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.295+0000 7fa50e015640 1 -- 172.21.15.37:0/929340306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5080d26f0 msgr2=0x7fa5080cfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.295+0000 7fa50e015640 1 --2- 172.21.15.37:0/929340306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5080d26f0 0x7fa5080cfd00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa508073710 tx=0x7fa50002a680 comp rx=0 tx=0).stop 2024-04-26T20:23:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.295+0000 7fa50e015640 1 -- 172.21.15.37:0/929340306 shutdown_connections 2024-04-26T20:23:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.295+0000 7fa50e015640 1 --2- 172.21.15.37:0/929340306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5080d26f0 0x7fa5080cfd00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.295+0000 7fa50e015640 1 --2- 172.21.15.37:0/929340306 >> v2:172.21.15.134:3300/0 conn(0x7fa5080b2290 0x7fa5080b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.295+0000 7fa50e015640 1 -- 172.21.15.37:0/929340306 >> 172.21.15.37:0/929340306 conn(0x7fa5080d2ff0 msgr2=0x7fa5080d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.295+0000 7fa50e015640 1 -- 172.21.15.37:0/929340306 shutdown_connections 2024-04-26T20:23:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.295+0000 7fa50e015640 1 -- 172.21.15.37:0/929340306 wait complete. 2024-04-26T20:23:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.296+0000 7fa50e015640 1 Processor -- start 2024-04-26T20:23:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.296+0000 7fa50e015640 1 -- start start 2024-04-26T20:23:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.297+0000 7fa50e015640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5080b2290 0x7fa5081596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.297+0000 7fa50e015640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5080cf870 0x7fa508158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.297+0000 7fa50e015640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5080d26f0 0x7fa5080b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.297+0000 7fa50e015640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa5080b2670 con 0x7fa5080d26f0 2024-04-26T20:23:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.297+0000 7fa50e015640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5080ce1c0 con 0x7fa5080b2290 2024-04-26T20:23:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.297+0000 7fa50e015640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa50815ec60 con 0x7fa5080cf870 2024-04-26T20:23:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.297+0000 7fa507fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5080d26f0 0x7fa5080b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.297+0000 7fa507fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5080d26f0 0x7fa5080b69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37088/0 (socket says 172.21.15.37:37088) 2024-04-26T20:23:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.297+0000 7fa507fff640 1 -- 172.21.15.37:0/3168507450 learned_addr learned my addr 172.21.15.37:0/3168507450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.297+0000 7fa5077fe640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5080b2290 0x7fa5081596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.297+0000 7fa506ffd640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5080cf870 0x7fa508158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa507fff640 1 -- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5080cf870 msgr2=0x7fa508158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa507fff640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5080cf870 0x7fa508158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa507fff640 1 -- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5080b2290 msgr2=0x7fa5081596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa507fff640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5080b2290 0x7fa5081596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa507fff640 1 -- 172.21.15.37:0/3168507450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5080c8c10 con 0x7fa5080d26f0 2024-04-26T20:23:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa506ffd640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5080cf870 0x7fa508158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa507fff640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5080d26f0 0x7fa5080b69c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa500009e50 tx=0x7fa500004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa4effff640 1 -- 172.21.15.37:0/3168507450 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50003f070 con 0x7fa5080d26f0 2024-04-26T20:23:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa4effff640 1 -- 172.21.15.37:0/3168507450 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa500031cc0 con 0x7fa5080d26f0 2024-04-26T20:23:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa4effff640 1 -- 172.21.15.37:0/3168507450 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500044680 con 0x7fa5080d26f0 2024-04-26T20:23:51.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa50e015640 1 -- 172.21.15.37:0/3168507450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5080c8930 con 0x7fa5080d26f0 2024-04-26T20:23:51.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.298+0000 7fa50e015640 1 -- 172.21.15.37:0/3168507450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5080b61c0 con 0x7fa5080d26f0 2024-04-26T20:23:51.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.299+0000 7fa50e015640 1 -- 172.21.15.37:0/3168507450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa50806a360 con 0x7fa5080d26f0 2024-04-26T20:23:51.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.300+0000 7fa4effff640 1 -- 172.21.15.37:0/3168507450 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fa50003b410 con 0x7fa5080d26f0 2024-04-26T20:23:51.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.300+0000 7fa4effff640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa4f406cbc0 0x7fa4f406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.300+0000 7fa4effff640 1 -- 172.21.15.37:0/3168507450 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7fa5000ad990 con 0x7fa5080d26f0 2024-04-26T20:23:51.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.300+0000 7fa5077fe640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa4f406cbc0 0x7fa4f406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.301+0000 7fa5077fe640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa4f406cbc0 0x7fa4f406f080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800a990 tx=0x7fa4f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:51.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.304+0000 7fa4effff640 1 -- 172.21.15.37:0/3168507450 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa50007ba90 con 0x7fa5080d26f0 2024-04-26T20:23:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.493+0000 7fa50e015640 1 -- 172.21.15.37:0/3168507450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa5080d05d0 con 0x7fa5080d26f0 2024-04-26T20:23:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.493+0000 7fa4effff640 1 -- 172.21.15.37:0/3168507450 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7fa50007b390 con 0x7fa5080d26f0 2024-04-26T20:23:51.495 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:23:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.495+0000 7fa50e015640 1 -- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa4f406cbc0 msgr2=0x7fa4f406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.495+0000 7fa50e015640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa4f406cbc0 0x7fa4f406f080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800a990 tx=0x7fa4f8008040 comp rx=0 tx=0).stop 2024-04-26T20:23:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.496+0000 7fa50e015640 1 -- 172.21.15.37:0/3168507450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5080d26f0 msgr2=0x7fa5080b69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.496+0000 7fa50e015640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5080d26f0 0x7fa5080b69c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa500009e50 tx=0x7fa500004a40 comp rx=0 tx=0).stop 2024-04-26T20:23:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.496+0000 7fa50e015640 1 -- 172.21.15.37:0/3168507450 shutdown_connections 2024-04-26T20:23:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.496+0000 7fa50e015640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5080d26f0 0x7fa5080b69c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.496+0000 7fa50e015640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa4f406cbc0 0x7fa4f406f080 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.496+0000 7fa50e015640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5080cf870 0x7fa508158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.496+0000 7fa50e015640 1 --2- 172.21.15.37:0/3168507450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5080b2290 0x7fa5081596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.496+0000 7fa50e015640 1 -- 172.21.15.37:0/3168507450 >> 172.21.15.37:0/3168507450 conn(0x7fa5080d2ff0 msgr2=0x7fa5080d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:51.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.496+0000 7fa50e015640 1 -- 172.21.15.37:0/3168507450 shutdown_connections 2024-04-26T20:23:51.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.496+0000 7fa50e015640 1 -- 172.21.15.37:0/3168507450 wait complete. 2024-04-26T20:23:51.508 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:23:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.608+0000 7fb502820640 1 Processor -- start 2024-04-26T20:23:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.608+0000 7fb502820640 1 -- start start 2024-04-26T20:23:51.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb502820640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4fc15f380 0x7fb4fc0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb502820640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb4fc0b9a90 0x7fb4fc0d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb502820640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4fc0d3de0 con 0x7fb4fc15f760 2024-04-26T20:23:51.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb502820640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb4fc0d43a0 con 0x7fb4fc0b9a90 2024-04-26T20:23:51.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb502820640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb4fc0d40c0 con 0x7fb4fc15f380 2024-04-26T20:23:51.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb500dbd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb4fc0b9a90 0x7fb4fc0d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb4fb7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb4fc15f760 0x7fb4fc0cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53832/0 (socket says 172.21.15.37:53832) 2024-04-26T20:23:51.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb500dbd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb4fc0b9a90 0x7fb4fc0d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37094/0 (socket says 172.21.15.37:37094) 2024-04-26T20:23:51.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb4fb7fe640 1 -- 172.21.15.37:0/3718347882 learned_addr learned my addr 172.21.15.37:0/3718347882 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:51.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb4fbfff640 1 --2- 172.21.15.37:0/3718347882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4fc15f380 0x7fb4fc0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb500dbd640 1 -- 172.21.15.37:0/3718347882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4fc15f380 msgr2=0x7fb4fc0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb500dbd640 1 --2- 172.21.15.37:0/3718347882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4fc15f380 0x7fb4fc0d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb500dbd640 1 -- 172.21.15.37:0/3718347882 >> v1:172.21.15.37:6789/0 conn(0x7fb4fc15f760 legacy=0x7fb4fc0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb500dbd640 1 -- 172.21.15.37:0/3718347882 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4fc0b8280 con 0x7fb4fc0b9a90 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb4faffd640 1 -- 172.21.15.37:0/3718347882 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 248958332 0 0) 0x7fb4fc0d3de0 con 0x7fb4fc15f760 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.609+0000 7fb4fbfff640 1 --2- 172.21.15.37:0/3718347882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4fc15f380 0x7fb4fc0d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.610+0000 7fb500dbd640 1 --2- 172.21.15.37:0/3718347882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4fc0b9a90 0x7fb4fc0d2110 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009d90 tx=0x7fb4f402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12da4c20280eb63d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.610+0000 7fb4faffd640 1 -- 172.21.15.37:0/3718347882 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4007740 con 0x7fb4fc0b9a90 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.610+0000 7fb4faffd640 1 -- 172.21.15.37:0/3718347882 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f40078e0 con 0x7fb4fc0b9a90 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.610+0000 7fb4faffd640 1 -- 172.21.15.37:0/3718347882 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4007cb0 con 0x7fb4fc0b9a90 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.611+0000 7fb502820640 1 -- 172.21.15.37:0/3718347882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4fc0b9a90 msgr2=0x7fb4fc0d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.611+0000 7fb502820640 1 --2- 172.21.15.37:0/3718347882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4fc0b9a90 0x7fb4fc0d2110 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009d90 tx=0x7fb4f402a6f0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.611+0000 7fb502820640 1 -- 172.21.15.37:0/3718347882 shutdown_connections 2024-04-26T20:23:51.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.611+0000 7fb502820640 1 --2- 172.21.15.37:0/3718347882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4fc0b9a90 0x7fb4fc0d2110 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.611+0000 7fb502820640 1 --2- 172.21.15.37:0/3718347882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4fc15f380 0x7fb4fc0d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.611+0000 7fb502820640 1 -- 172.21.15.37:0/3718347882 >> 172.21.15.37:0/3718347882 conn(0x7fb4fc09e560 msgr2=0x7fb4fc0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:51.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.611+0000 7fb502820640 1 -- 172.21.15.37:0/3718347882 shutdown_connections 2024-04-26T20:23:51.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.611+0000 7fb502820640 1 -- 172.21.15.37:0/3718347882 wait complete. 2024-04-26T20:23:51.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.612+0000 7fb502820640 1 Processor -- start 2024-04-26T20:23:51.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.612+0000 7fb502820640 1 -- start start 2024-04-26T20:23:51.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb502820640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4fc0b9a90 0x7fb4fc15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb502820640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4fc15f380 0x7fb4fc1f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb502820640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4fc15f760 0x7fb4fc1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb502820640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc0d1280 con 0x7fb4fc15f760 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb502820640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc0d1100 con 0x7fb4fc15f380 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb502820640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb4fc0d5b50 con 0x7fb4fc0b9a90 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb500dbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4fc15f760 0x7fb4fc1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb500dbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4fc15f760 0x7fb4fc1597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37102/0 (socket says 172.21.15.37:37102) 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb500dbd640 1 -- 172.21.15.37:0/311413963 learned_addr learned my addr 172.21.15.37:0/311413963 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb4fb7fe640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4fc15f380 0x7fb4fc1f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb4fbfff640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4fc0b9a90 0x7fb4fc15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb500dbd640 1 -- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4fc0b9a90 msgr2=0x7fb4fc15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb500dbd640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4fc0b9a90 0x7fb4fc15b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb500dbd640 1 -- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4fc15f380 msgr2=0x7fb4fc1f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb500dbd640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4fc15f380 0x7fb4fc1f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb500dbd640 1 -- 172.21.15.37:0/311413963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4fc1f5600 con 0x7fb4fc15f760 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb500dbd640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4fc15f760 0x7fb4fc1597a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009eb0 tx=0x7fb4f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.613+0000 7fb4f8ff9640 1 -- 172.21.15.37:0/311413963 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4040070 con 0x7fb4fc15f760 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.614+0000 7fb4f8ff9640 1 -- 172.21.15.37:0/311413963 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f4031bc0 con 0x7fb4fc15f760 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.614+0000 7fb4f8ff9640 1 -- 172.21.15.37:0/311413963 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4045460 con 0x7fb4fc15f760 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.614+0000 7fb502820640 1 -- 172.21.15.37:0/311413963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4fc1f58e0 con 0x7fb4fc15f760 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.614+0000 7fb502820640 1 -- 172.21.15.37:0/311413963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4fc155430 con 0x7fb4fc15f760 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.615+0000 7fb502820640 1 -- 172.21.15.37:0/311413963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4fc06a360 con 0x7fb4fc15f760 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.615+0000 7fb4f8ff9640 1 -- 172.21.15.37:0/311413963 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fb4f4039040 con 0x7fb4fc15f760 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.616+0000 7fb4f8ff9640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb4e806cbc0 0x7fb4e806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.616+0000 7fb4f8ff9640 1 -- 172.21.15.37:0/311413963 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7fb4f40ad8b0 con 0x7fb4fc15f760 2024-04-26T20:23:51.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.616+0000 7fb4fbfff640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb4e806cbc0 0x7fb4e806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.616+0000 7fb4fbfff640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb4e806cbc0 0x7fb4e806f080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0006fd0 tx=0x7fb4f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:51.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.620+0000 7fb4f8ff9640 1 -- 172.21.15.37:0/311413963 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4f407b210 con 0x7fb4fc15f760 2024-04-26T20:23:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.815+0000 7fb502820640 1 -- 172.21.15.37:0/311413963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb4fc06eb00 con 0x7fb4fc15f760 2024-04-26T20:23:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.816+0000 7fb4f8ff9640 1 -- 172.21.15.37:0/311413963 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7fb4f407f510 con 0x7fb4fc15f760 2024-04-26T20:23:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-26T20:23:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 -- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb4e806cbc0 msgr2=0x7fb4e806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb4e806cbc0 0x7fb4e806f080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0006fd0 tx=0x7fb4f0008040 comp rx=0 tx=0).stop 2024-04-26T20:23:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 -- 172.21.15.37:0/311413963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4fc15f760 msgr2=0x7fb4fc1597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4fc15f760 0x7fb4fc1597a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009eb0 tx=0x7fb4f4004a40 comp rx=0 tx=0).stop 2024-04-26T20:23:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 -- 172.21.15.37:0/311413963 shutdown_connections 2024-04-26T20:23:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4fc15f760 0x7fb4fc1597a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4fc15f380 0x7fb4fc1f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb4e806cbc0 0x7fb4e806f080 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 --2- 172.21.15.37:0/311413963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4fc0b9a90 0x7fb4fc15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 -- 172.21.15.37:0/311413963 >> 172.21.15.37:0/311413963 conn(0x7fb4fc09e560 msgr2=0x7fb4fc06d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 -- 172.21.15.37:0/311413963 shutdown_connections 2024-04-26T20:23:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:51.818+0000 7fb502820640 1 -- 172.21.15.37:0/311413963 wait complete. 2024-04-26T20:23:51.830 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T20:23:51.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:51.830 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T20:23:51.875 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:23:51.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:51.875 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:23:51.900 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:23:51.900 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:23:51.900 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:51.917 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:23:51.917 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:51.974 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:23:52.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.076+0000 7f0c21fe4640 1 Processor -- start 2024-04-26T20:23:52.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.076+0000 7f0c21fe4640 1 -- start start 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c21fe4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0c1c15fc10 0x7f0c1c1600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c21fe4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c1c072e30 con 0x7f0c1c0ce9b0 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c21fe4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c1c073110 con 0x7f0c1c159a90 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c21fe4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f0c1c0733f0 con 0x7f0c1c15fc10 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c1bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0c1c0ce9b0 0x7f0c1c0cee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53836/0 (socket says 172.21.15.37:53836) 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c1bfff640 1 -- 172.21.15.37:0/4140210948 learned_addr learned my addr 172.21.15.37:0/4140210948 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c1affd640 1 --2- 172.21.15.37:0/4140210948 >> v2:172.21.15.134:3300/0 conn(0x7f0c1c15fc10 0x7f0c1c1600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c1a7fc640 1 -- 172.21.15.37:0/4140210948 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2600986815 0 0) 0x7f0c1c072e30 con 0x7f0c1c0ce9b0 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c1a7fc640 1 -- 172.21.15.37:0/4140210948 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c04003520 con 0x7f0c1c0ce9b0 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c1a7fc640 1 -- 172.21.15.37:0/4140210948 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3329936329 0 0) 0x7f0c04003520 con 0x7f0c1c0ce9b0 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c1affd640 1 -- 172.21.15.37:0/4140210948 >> v1:172.21.15.134:6789/0 conn(0x7f0c1c159a90 legacy=0x7f0c1c06fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c1affd640 1 -- 172.21.15.37:0/4140210948 >> v1:172.21.15.37:6789/0 conn(0x7f0c1c0ce9b0 legacy=0x7f0c1c0cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.077+0000 7f0c1affd640 1 -- 172.21.15.37:0/4140210948 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c0736d0 con 0x7f0c1c15fc10 2024-04-26T20:23:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.078+0000 7f0c1affd640 1 --2- 172.21.15.37:0/4140210948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c15fc10 0x7f0c1c1600c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c100097a0 tx=0x7f0c10029fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c69c1462478f668a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.078+0000 7f0c1a7fc640 1 -- 172.21.15.37:0/4140210948 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10037070 con 0x7f0c1c15fc10 2024-04-26T20:23:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.078+0000 7f0c1a7fc640 1 -- 172.21.15.37:0/4140210948 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1002aa40 con 0x7f0c1c15fc10 2024-04-26T20:23:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.078+0000 7f0c1a7fc640 1 -- 172.21.15.37:0/4140210948 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140019f0 con 0x7f0c1c15fc10 2024-04-26T20:23:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.079+0000 7f0c21fe4640 1 -- 172.21.15.37:0/4140210948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c15fc10 msgr2=0x7f0c1c1600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.079+0000 7f0c21fe4640 1 --2- 172.21.15.37:0/4140210948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c15fc10 0x7f0c1c1600c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c100097a0 tx=0x7f0c10029fb0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.079+0000 7f0c21fe4640 1 -- 172.21.15.37:0/4140210948 shutdown_connections 2024-04-26T20:23:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.079+0000 7f0c21fe4640 1 --2- 172.21.15.37:0/4140210948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c15fc10 0x7f0c1c1600c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.079+0000 7f0c21fe4640 1 -- 172.21.15.37:0/4140210948 >> 172.21.15.37:0/4140210948 conn(0x7f0c1c0d0d50 msgr2=0x7f0c1c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.079+0000 7f0c21fe4640 1 -- 172.21.15.37:0/4140210948 shutdown_connections 2024-04-26T20:23:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.079+0000 7f0c21fe4640 1 -- 172.21.15.37:0/4140210948 wait complete. 2024-04-26T20:23:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.080+0000 7f0c21fe4640 1 Processor -- start 2024-04-26T20:23:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.080+0000 7f0c21fe4640 1 -- start start 2024-04-26T20:23:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.081+0000 7f0c21fe4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c0ce9b0 0x7f0c1c15d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.081+0000 7f0c21fe4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c159a90 0x7f0c1c158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.081+0000 7f0c21fe4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c1c15fc10 0x7f0c1c158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.081+0000 7f0c21fe4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c159e70 con 0x7f0c1c15fc10 2024-04-26T20:23:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.081+0000 7f0c21fe4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c070300 con 0x7f0c1c159a90 2024-04-26T20:23:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.081+0000 7f0c21fe4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1c159ff0 con 0x7f0c1c0ce9b0 2024-04-26T20:23:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.081+0000 7f0c1bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c1c15fc10 0x7f0c1c158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.081+0000 7f0c1bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c1c15fc10 0x7f0c1c158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37108/0 (socket says 172.21.15.37:37108) 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.081+0000 7f0c1bfff640 1 -- 172.21.15.37:0/15645314 learned_addr learned my addr 172.21.15.37:0/15645314 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.081+0000 7f0c1affd640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c159a90 0x7f0c1c158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.081+0000 7f0c1b7fe640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c0ce9b0 0x7f0c1c15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c1bfff640 1 -- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c0ce9b0 msgr2=0x7f0c1c15d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c1bfff640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c0ce9b0 0x7f0c1c15d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c1bfff640 1 -- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c159a90 msgr2=0x7f0c1c158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c1bfff640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c159a90 0x7f0c1c158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c1bfff640 1 -- 172.21.15.37:0/15645314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c1f7790 con 0x7f0c1c15fc10 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c1affd640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c159a90 0x7f0c1c158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c1bfff640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c1c15fc10 0x7f0c1c158ca0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0c14001560 tx=0x7f0c1400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c03fff640 1 -- 172.21.15.37:0/15645314 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14018070 con 0x7f0c1c15fc10 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c03fff640 1 -- 172.21.15.37:0/15645314 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1400a6a0 con 0x7f0c1c15fc10 2024-04-26T20:23:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c21fe4640 1 -- 172.21.15.37:0/15645314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1c15a4b0 con 0x7f0c1c15fc10 2024-04-26T20:23:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c03fff640 1 -- 172.21.15.37:0/15645314 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1400aa90 con 0x7f0c1c15fc10 2024-04-26T20:23:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.082+0000 7f0c21fe4640 1 -- 172.21.15.37:0/15645314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1c15aa70 con 0x7f0c1c15fc10 2024-04-26T20:23:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.083+0000 7f0c21fe4640 1 -- 172.21.15.37:0/15645314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1c06a360 con 0x7f0c1c15fc10 2024-04-26T20:23:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.083+0000 7f0c03fff640 1 -- 172.21.15.37:0/15645314 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f0c1400ac30 con 0x7f0c1c15fc10 2024-04-26T20:23:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.084+0000 7f0c03fff640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0c0806cbc0 0x7f0c0806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.084+0000 7f0c03fff640 1 -- 172.21.15.37:0/15645314 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f0c1408b750 con 0x7f0c1c15fc10 2024-04-26T20:23:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.084+0000 7f0c1b7fe640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0c0806cbc0 0x7f0c0806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.085+0000 7f0c1b7fe640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0c0806cbc0 0x7f0c0806f080 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c001840 tx=0x7f0c0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.088+0000 7f0c03fff640 1 -- 172.21.15.37:0/15645314 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0c140597d0 con 0x7f0c1c15fc10 2024-04-26T20:23:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.274+0000 7f0c21fe4640 1 -- 172.21.15.37:0/15645314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c1c158210 con 0x7f0c1c15fc10 2024-04-26T20:23:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.275+0000 7f0c03fff640 1 -- 172.21.15.37:0/15645314 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16746 (secure 0 0 0) 0x7f0c140590d0 con 0x7f0c1c15fc10 2024-04-26T20:23:52.279 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:23:52.279 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:23:34.988080+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:22:39.762703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:22:41.066360+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:23:52.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 -- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0c0806cbc0 msgr2=0x7f0c0806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0c0806cbc0 0x7f0c0806f080 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c001840 tx=0x7f0c0c009040 comp rx=0 tx=0).stop 2024-04-26T20:23:52.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 -- 172.21.15.37:0/15645314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c1c15fc10 msgr2=0x7f0c1c158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c1c15fc10 0x7f0c1c158ca0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0c14001560 tx=0x7f0c1400cf50 comp rx=0 tx=0).stop 2024-04-26T20:23:52.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 -- 172.21.15.37:0/15645314 shutdown_connections 2024-04-26T20:23:52.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c1c15fc10 0x7f0c1c158ca0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c159a90 0x7f0c1c158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0c0806cbc0 0x7f0c0806f080 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 --2- 172.21.15.37:0/15645314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c0ce9b0 0x7f0c1c15d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 -- 172.21.15.37:0/15645314 >> 172.21.15.37:0/15645314 conn(0x7f0c1c0d0d50 msgr2=0x7f0c1c0cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:52.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 -- 172.21.15.37:0/15645314 shutdown_connections 2024-04-26T20:23:52.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.277+0000 7f0c21fe4640 1 -- 172.21.15.37:0/15645314 wait complete. 2024-04-26T20:23:52.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:22:39.762703+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:22:41.066360+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:23:52.290 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:23:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.392+0000 7fcf152b1640 1 Processor -- start 2024-04-26T20:23:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.392+0000 7fcf152b1640 1 -- start start 2024-04-26T20:23:52.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.393+0000 7fcf152b1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcf100d21b0 0x7fcf1016a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.393+0000 7fcf152b1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf100d2ed0 con 0x7fcf100d0f00 2024-04-26T20:23:52.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.393+0000 7fcf152b1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf100d31b0 con 0x7fcf1015f470 2024-04-26T20:23:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.393+0000 7fcf152b1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcf100d3490 con 0x7fcf100d21b0 2024-04-26T20:23:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.393+0000 7fcf0e7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fcf100d0f00 0x7fcf100d1aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53840/0 (socket says 172.21.15.37:53840) 2024-04-26T20:23:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.393+0000 7fcf0e7fc640 1 -- 172.21.15.37:0/3155251748 learned_addr learned my addr 172.21.15.37:0/3155251748 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.393+0000 7fcf0f7fe640 1 --2- 172.21.15.37:0/3155251748 >> v2:172.21.15.134:3300/0 conn(0x7fcf100d21b0 0x7fcf1016a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.393+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728878110 0 0) 0x7fcf100d2ed0 con 0x7fcf100d0f00 2024-04-26T20:23:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.393+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcefc003520 con 0x7fcf100d0f00 2024-04-26T20:23:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4086772699 0 0) 0x7fcefc003520 con 0x7fcf100d0f00 2024-04-26T20:23:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 >> v2:172.21.15.134:3300/0 conn(0x7fcf100d21b0 msgr2=0x7fcf1016a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 --2- 172.21.15.37:0/3155251748 >> v2:172.21.15.134:3300/0 conn(0x7fcf100d21b0 0x7fcf1016a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 >> v1:172.21.15.134:6789/0 conn(0x7fcf1015f470 legacy=0x7fcf100d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf100d3770 con 0x7fcf100d0f00 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fcef8001670 con 0x7fcf100d0f00 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 >> v1:172.21.15.37:6789/0 conn(0x7fcf100d0f00 legacy=0x7fcf100d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 --2- 172.21.15.37:0/3155251748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcefc0039b0 0x7fcefc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 --2- 172.21.15.37:0/3155251748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcefc004800 0x7fcefc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 --2- 172.21.15.37:0/3155251748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcefc0051c0 0x7fcefc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf100d3490 con 0x7fcefc0039b0 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc00c060 con 0x7fcefc004800 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcefc00c260 con 0x7fcefc0051c0 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0effd640 1 --2- 172.21.15.37:0/3155251748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcefc0039b0 0x7fcefc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0e7fc640 1 --2- 172.21.15.37:0/3155251748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcefc004800 0x7fcefc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0f7fe640 1 --2- 172.21.15.37:0/3155251748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcefc0051c0 0x7fcefc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0effd640 1 -- 172.21.15.37:0/3155251748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcefc0051c0 msgr2=0x7fcefc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0effd640 1 --2- 172.21.15.37:0/3155251748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcefc0051c0 0x7fcefc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0effd640 1 -- 172.21.15.37:0/3155251748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcefc004800 msgr2=0x7fcefc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0effd640 1 --2- 172.21.15.37:0/3155251748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcefc004800 0x7fcefc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0effd640 1 -- 172.21.15.37:0/3155251748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc003520 con 0x7fcefc0039b0 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.394+0000 7fcf0f7fe640 1 -- 172.21.15.37:0/3155251748 reap_dead start 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.395+0000 7fcf0effd640 1 --2- 172.21.15.37:0/3155251748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcefc0039b0 0x7fcefc003dd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcf08001750 tx=0x7fcefc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61f1bdc3042fd4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.395+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08001370 con 0x7fcefc0039b0 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.395+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf080052d0 con 0x7fcefc0039b0 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.395+0000 7fcf0dffb640 1 -- 172.21.15.37:0/3155251748 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0802abe0 con 0x7fcefc0039b0 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.396+0000 7fcf152b1640 1 -- 172.21.15.37:0/3155251748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcefc0039b0 msgr2=0x7fcefc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.396+0000 7fcf152b1640 1 --2- 172.21.15.37:0/3155251748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcefc0039b0 0x7fcefc003dd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcf08001750 tx=0x7fcefc005830 comp rx=0 tx=0).stop 2024-04-26T20:23:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.396+0000 7fcf152b1640 1 -- 172.21.15.37:0/3155251748 shutdown_connections 2024-04-26T20:23:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.396+0000 7fcf152b1640 1 --2- 172.21.15.37:0/3155251748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcefc0039b0 0x7fcefc003dd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.396+0000 7fcf152b1640 1 -- 172.21.15.37:0/3155251748 >> 172.21.15.37:0/3155251748 conn(0x7fcf1009e510 msgr2=0x7fcf100d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.396+0000 7fcf152b1640 1 -- 172.21.15.37:0/3155251748 shutdown_connections 2024-04-26T20:23:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.396+0000 7fcf152b1640 1 -- 172.21.15.37:0/3155251748 wait complete. 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.397+0000 7fcf152b1640 1 Processor -- start 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.397+0000 7fcf152b1640 1 -- start start 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf152b1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcefc0039b0 0x7fcf1015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf152b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf100b8ce0 0x7fcf100ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf152b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf101f6160 0x7fcf101611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf152b1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf100d3f70 con 0x7fcf101f6160 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf152b1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1015f850 con 0x7fcf100b8ce0 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf152b1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcf100b9160 con 0x7fcefc0039b0 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf101f6160 0x7fcf101611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf101f6160 0x7fcf101611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37118/0 (socket says 172.21.15.37:37118) 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0f7fe640 1 -- 172.21.15.37:0/3359301448 learned_addr learned my addr 172.21.15.37:0/3359301448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0effd640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcefc0039b0 0x7fcf1015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0e7fc640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf100b8ce0 0x7fcf100ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0f7fe640 1 -- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcefc0039b0 msgr2=0x7fcf1015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0f7fe640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcefc0039b0 0x7fcf1015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0f7fe640 1 -- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf100b8ce0 msgr2=0x7fcf100ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0f7fe640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf100b8ce0 0x7fcf100ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0f7fe640 1 -- 172.21.15.37:0/3359301448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf100b93e0 con 0x7fcf101f6160 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0effd640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcefc0039b0 0x7fcf1015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0e7fc640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf100b8ce0 0x7fcf100ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf0f7fe640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf101f6160 0x7fcf101611b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcf10164070 tx=0x7fcf0000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.398+0000 7fcf077fe640 1 -- 172.21.15.37:0/3359301448 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000fce0 con 0x7fcf101f6160 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.399+0000 7fcf077fe640 1 -- 172.21.15.37:0/3359301448 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0000fe80 con 0x7fcf101f6160 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.399+0000 7fcf152b1640 1 -- 172.21.15.37:0/3359301448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf100b96c0 con 0x7fcf101f6160 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.399+0000 7fcf077fe640 1 -- 172.21.15.37:0/3359301448 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf00015670 con 0x7fcf101f6160 2024-04-26T20:23:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.399+0000 7fcf152b1640 1 -- 172.21.15.37:0/3359301448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf100cc2d0 con 0x7fcf101f6160 2024-04-26T20:23:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.400+0000 7fcf152b1640 1 -- 172.21.15.37:0/3359301448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1006a360 con 0x7fcf101f6160 2024-04-26T20:23:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.400+0000 7fcf077fe640 1 -- 172.21.15.37:0/3359301448 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fcf00015810 con 0x7fcf101f6160 2024-04-26T20:23:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.400+0000 7fcf077fe640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcef006cbc0 0x7fcef006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.401+0000 7fcf077fe640 1 -- 172.21.15.37:0/3359301448 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7fcf0008ceb0 con 0x7fcf101f6160 2024-04-26T20:23:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.401+0000 7fcf0effd640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcef006cbc0 0x7fcef006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.401+0000 7fcf0effd640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcef006cbc0 0x7fcef006f080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf08032000 tx=0x7fcf0802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:52.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.405+0000 7fcf077fe640 1 -- 172.21.15.37:0/3359301448 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf00019030 con 0x7fcf101f6160 2024-04-26T20:23:52.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.589+0000 7fcf152b1640 1 -- 172.21.15.37:0/3359301448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf100d5870 con 0x7fcf101f6160 2024-04-26T20:23:52.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.590+0000 7fcf077fe640 1 -- 172.21.15.37:0/3359301448 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7fcf00010040 con 0x7fcf101f6160 2024-04-26T20:23:52.591 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:23:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 -- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcef006cbc0 msgr2=0x7fcef006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcef006cbc0 0x7fcef006f080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf08032000 tx=0x7fcf0802e000 comp rx=0 tx=0).stop 2024-04-26T20:23:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 -- 172.21.15.37:0/3359301448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf101f6160 msgr2=0x7fcf101611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf101f6160 0x7fcf101611b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcf10164070 tx=0x7fcf0000f1b0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 -- 172.21.15.37:0/3359301448 shutdown_connections 2024-04-26T20:23:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf101f6160 0x7fcf101611b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf100b8ce0 0x7fcf100ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcef006cbc0 0x7fcef006f080 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 --2- 172.21.15.37:0/3359301448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcefc0039b0 0x7fcf1015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 -- 172.21.15.37:0/3359301448 >> 172.21.15.37:0/3359301448 conn(0x7fcf1009e510 msgr2=0x7fcf1015d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 -- 172.21.15.37:0/3359301448 shutdown_connections 2024-04-26T20:23:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.592+0000 7fcf152b1640 1 -- 172.21.15.37:0/3359301448 wait complete. 2024-04-26T20:23:52.604 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:23:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.706+0000 7f08665ac640 1 Processor -- start 2024-04-26T20:23:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.706+0000 7f08665ac640 1 -- start start 2024-04-26T20:23:52.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f08665ac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600cfba0 0x7f08600b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f08665ac640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08600cdb90 con 0x7f086015d830 2024-04-26T20:23:52.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f08665ac640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08600d1ff0 con 0x7f08600b25e0 2024-04-26T20:23:52.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f08665ac640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f08600b2390 con 0x7f08600cfba0 2024-04-26T20:23:52.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f0864b49640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f086015d830 0x7f08600ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53856/0 (socket says 172.21.15.37:53856) 2024-04-26T20:23:52.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f0864b49640 1 -- 172.21.15.37:0/2964473430 learned_addr learned my addr 172.21.15.37:0/2964473430 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:52.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085ffff640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600cfba0 0x7f08600b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3360609540 0 0) 0x7f08600cdb90 con 0x7f086015d830 2024-04-26T20:23:52.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0848003520 con 0x7f086015d830 2024-04-26T20:23:52.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1903705477 0 0) 0x7f0848003520 con 0x7f086015d830 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600cfba0 msgr2=0x7f08600b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600cfba0 0x7f08600b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 >> v1:172.21.15.134:6789/0 conn(0x7f08600b25e0 legacy=0x7f086015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08600d1a30 con 0x7f086015d830 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f08580019f0 con 0x7f086015d830 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 >> v1:172.21.15.37:6789/0 conn(0x7f086015d830 legacy=0x7f08600ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600cfba0 0x7f08480039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 0x7f0848004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0848004d40 0x7f08480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f084800bb80 con 0x7f0848003fb0 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f084800bd30 con 0x7f0848004d40 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f084800bf30 con 0x7f08600cfba0 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.707+0000 7f085f7fe640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 0x7f0848004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.708+0000 7f085ffff640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600cfba0 0x7f08480039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.708+0000 7f0864b49640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0848004d40 0x7f08480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.708+0000 7f085f7fe640 1 -- 172.21.15.37:0/2964473430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600cfba0 msgr2=0x7f08480039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.708+0000 7f085f7fe640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600cfba0 0x7f08480039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.708+0000 7f085f7fe640 1 -- 172.21.15.37:0/2964473430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0848004d40 msgr2=0x7f08480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.708+0000 7f085f7fe640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0848004d40 0x7f08480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.708+0000 7f085f7fe640 1 -- 172.21.15.37:0/2964473430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0848003520 con 0x7f0848003fb0 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.708+0000 7f085f7fe640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 0x7f0848004800 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0854001750 tx=0x7f0848005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d215344f93ec07a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.709+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854001370 con 0x7f0848003fb0 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.709+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08540052d0 con 0x7f0848003fb0 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.709+0000 7f085effd640 1 -- 172.21.15.37:0/2964473430 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085402abe0 con 0x7f0848003fb0 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.709+0000 7f08665ac640 1 -- 172.21.15.37:0/2964473430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 msgr2=0x7f0848004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.709+0000 7f08665ac640 1 --2- 172.21.15.37:0/2964473430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 0x7f0848004800 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0854001750 tx=0x7f0848005830 comp rx=0 tx=0).stop 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.709+0000 7f0864b49640 1 -- 172.21.15.37:0/2964473430 reap_dead start 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.709+0000 7f08665ac640 1 -- 172.21.15.37:0/2964473430 shutdown_connections 2024-04-26T20:23:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.709+0000 7f08665ac640 1 -- 172.21.15.37:0/2964473430 >> 172.21.15.37:0/2964473430 conn(0x7f08600d2f80 msgr2=0x7f08600d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:52.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.709+0000 7f08665ac640 1 -- 172.21.15.37:0/2964473430 shutdown_connections 2024-04-26T20:23:52.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.709+0000 7f08665ac640 1 -- 172.21.15.37:0/2964473430 wait complete. 2024-04-26T20:23:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.710+0000 7f08665ac640 1 Processor -- start 2024-04-26T20:23:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.711+0000 7f08665ac640 1 -- start start 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.711+0000 7f08665ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 0x7f08600c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.711+0000 7f08665ac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600b64c0 0x7f08601554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.711+0000 7f08665ac640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08600b7e20 0x7f08600b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.711+0000 7f08665ac640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08600b4320 con 0x7f0848003fb0 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.711+0000 7f08665ac640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08600cde30 con 0x7f08600b7e20 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.711+0000 7f08665ac640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f08600b44a0 con 0x7f08600b64c0 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.711+0000 7f085ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 0x7f08600c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.711+0000 7f085ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 0x7f08600c59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37144/0 (socket says 172.21.15.37:37144) 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.711+0000 7f085ffff640 1 -- 172.21.15.37:0/1044265869 learned_addr learned my addr 172.21.15.37:0/1044265869 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f0864b49640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08600b7e20 0x7f08600b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f085f7fe640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600b64c0 0x7f08601554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f085ffff640 1 -- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600b64c0 msgr2=0x7f08601554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f085ffff640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600b64c0 0x7f08601554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f085ffff640 1 -- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08600b7e20 msgr2=0x7f08600b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f085ffff640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08600b7e20 0x7f08600b82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f085ffff640 1 -- 172.21.15.37:0/1044265869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08600b16c0 con 0x7f0848003fb0 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f085f7fe640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600b64c0 0x7f08601554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f085ffff640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 0x7f08600c59d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f08500119d0 tx=0x7f0850011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f085cff9640 1 -- 172.21.15.37:0/1044265869 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085000faf0 con 0x7f0848003fb0 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f085cff9640 1 -- 172.21.15.37:0/1044265869 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085000fc90 con 0x7f0848003fb0 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f08665ac640 1 -- 172.21.15.37:0/1044265869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08600ca050 con 0x7f0848003fb0 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f085cff9640 1 -- 172.21.15.37:0/1044265869 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850014440 con 0x7f0848003fb0 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.712+0000 7f08665ac640 1 -- 172.21.15.37:0/1044265869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086006d7d0 con 0x7f0848003fb0 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.713+0000 7f08665ac640 1 -- 172.21.15.37:0/1044265869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086006a360 con 0x7f0848003fb0 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.714+0000 7f085cff9640 1 -- 172.21.15.37:0/1044265869 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f0850012040 con 0x7f0848003fb0 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.714+0000 7f085cff9640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f084c06cbc0 0x7f084c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.714+0000 7f085cff9640 1 -- 172.21.15.37:0/1044265869 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f085008c580 con 0x7f0848003fb0 2024-04-26T20:23:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.714+0000 7f085f7fe640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f084c06cbc0 0x7f084c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.715+0000 7f085f7fe640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f084c06cbc0 0x7f084c06f080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0854032000 tx=0x7f085402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.718+0000 7f085cff9640 1 -- 172.21.15.37:0/1044265869 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0850019030 con 0x7f0848003fb0 2024-04-26T20:23:52.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.902+0000 7f08665ac640 1 -- 172.21.15.37:0/1044265869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f086015c600 con 0x7f0848003fb0 2024-04-26T20:23:52.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.903+0000 7f085cff9640 1 -- 172.21.15.37:0/1044265869 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f085005a640 con 0x7f0848003fb0 2024-04-26T20:23:52.904 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:23:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.904+0000 7f08665ac640 1 -- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f084c06cbc0 msgr2=0x7f084c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.904+0000 7f08665ac640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f084c06cbc0 0x7f084c06f080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0854032000 tx=0x7f085402e000 comp rx=0 tx=0).stop 2024-04-26T20:23:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.904+0000 7f08665ac640 1 -- 172.21.15.37:0/1044265869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 msgr2=0x7f08600c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.904+0000 7f08665ac640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 0x7f08600c59d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f08500119d0 tx=0x7f0850011e90 comp rx=0 tx=0).stop 2024-04-26T20:23:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.904+0000 7f08665ac640 1 -- 172.21.15.37:0/1044265869 shutdown_connections 2024-04-26T20:23:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.904+0000 7f08665ac640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08600b7e20 0x7f08600b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.904+0000 7f08665ac640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f084c06cbc0 0x7f084c06f080 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.904+0000 7f08665ac640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08600b64c0 0x7f08601554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.904+0000 7f08665ac640 1 --2- 172.21.15.37:0/1044265869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0848003fb0 0x7f08600c59d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.904+0000 7f08665ac640 1 -- 172.21.15.37:0/1044265869 >> 172.21.15.37:0/1044265869 conn(0x7f08600d2f80 msgr2=0x7f0860075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:52.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.905+0000 7f08665ac640 1 -- 172.21.15.37:0/1044265869 shutdown_connections 2024-04-26T20:23:52.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:52.905+0000 7f08665ac640 1 -- 172.21.15.37:0/1044265869 wait complete. 2024-04-26T20:23:52.916 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:23:53.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.019+0000 7fd9b2020640 1 Processor -- start 2024-04-26T20:23:53.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.019+0000 7fd9b2020640 1 -- start start 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9b2020640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd9ac15f8a0 0x7fd9ac0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9b2020640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd9ac0b4aa0 0x7fd9ac169010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9b2020640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9ac0b4eb0 con 0x7fd9ac160370 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9b2020640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fd9ac068780 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9b2020640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd9ac06ca20 con 0x7fd9ac0b4aa0 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9aaffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd9ac15f8a0 0x7fd9ac0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9aaffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd9ac15f8a0 0x7fd9ac0b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37152/0 (socket says 172.21.15.37:37152) 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9aaffd640 1 -- 172.21.15.37:0/4137950936 learned_addr learned my addr 172.21.15.37:0/4137950936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9abfff640 1 --2- 172.21.15.37:0/4137950936 >> v2:172.21.15.134:3300/0 conn(0x7fd9ac0b4aa0 0x7fd9ac169010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9aaffd640 1 -- 172.21.15.37:0/4137950936 >> v2:172.21.15.134:3300/0 conn(0x7fd9ac0b4aa0 msgr2=0x7fd9ac169010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9aaffd640 1 --2- 172.21.15.37:0/4137950936 >> v2:172.21.15.134:3300/0 conn(0x7fd9ac0b4aa0 0x7fd9ac169010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9aaffd640 1 -- 172.21.15.37:0/4137950936 >> v1:172.21.15.134:6789/0 conn(0x7fd9ac160370 legacy=0x7fd9ac06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.020+0000 7fd9aaffd640 1 -- 172.21.15.37:0/4137950936 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac06cd00 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.021+0000 7fd9aaffd640 1 --2- 172.21.15.37:0/4137950936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9ac15f8a0 0x7fd9ac0b4560 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0d2460 tx=0x7fd9a002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9273be69915518 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.021+0000 7fd9aa7fc640 1 -- 172.21.15.37:0/4137950936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0037070 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.021+0000 7fd9aa7fc640 1 -- 172.21.15.37:0/4137950936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a0032440 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.021+0000 7fd9aa7fc640 1 -- 172.21.15.37:0/4137950936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0032810 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.022+0000 7fd9b2020640 1 -- 172.21.15.37:0/4137950936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9ac15f8a0 msgr2=0x7fd9ac0b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.022+0000 7fd9b2020640 1 --2- 172.21.15.37:0/4137950936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9ac15f8a0 0x7fd9ac0b4560 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0d2460 tx=0x7fd9a002a230 comp rx=0 tx=0).stop 2024-04-26T20:23:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.022+0000 7fd9b2020640 1 -- 172.21.15.37:0/4137950936 shutdown_connections 2024-04-26T20:23:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.022+0000 7fd9b2020640 1 --2- 172.21.15.37:0/4137950936 >> v2:172.21.15.134:3300/0 conn(0x7fd9ac0b4aa0 0x7fd9ac169010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.022+0000 7fd9b2020640 1 --2- 172.21.15.37:0/4137950936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9ac15f8a0 0x7fd9ac0b4560 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.022+0000 7fd9b2020640 1 -- 172.21.15.37:0/4137950936 >> 172.21.15.37:0/4137950936 conn(0x7fd9ac0d16b0 msgr2=0x7fd9ac0d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:53.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.022+0000 7fd9b2020640 1 -- 172.21.15.37:0/4137950936 shutdown_connections 2024-04-26T20:23:53.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.022+0000 7fd9b2020640 1 -- 172.21.15.37:0/4137950936 wait complete. 2024-04-26T20:23:53.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.023+0000 7fd9b2020640 1 Processor -- start 2024-04-26T20:23:53.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.023+0000 7fd9b2020640 1 -- start start 2024-04-26T20:23:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9b2020640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9ac0b4aa0 0x7fd9ac159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9b2020640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9ac15f8a0 0x7fd9ac1344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9b2020640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ac160370 0x7fd9ac0c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9b2020640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac06d5a0 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9b2020640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac06d2c0 con 0x7fd9ac160370 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9b2020640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ac06dac0 con 0x7fd9ac0b4aa0 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9aaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9ac15f8a0 0x7fd9ac1344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9aaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9ac15f8a0 0x7fd9ac1344d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37162/0 (socket says 172.21.15.37:37162) 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9aaffd640 1 -- 172.21.15.37:0/2567411293 learned_addr learned my addr 172.21.15.37:0/2567411293 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9abfff640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ac160370 0x7fd9ac0c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9ab7fe640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9ac0b4aa0 0x7fd9ac159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9aaffd640 1 -- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9ac0b4aa0 msgr2=0x7fd9ac159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9aaffd640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9ac0b4aa0 0x7fd9ac159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9aaffd640 1 -- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ac160370 msgr2=0x7fd9ac0c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9aaffd640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ac160370 0x7fd9ac0c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9aaffd640 1 -- 172.21.15.37:0/2567411293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac15a5d0 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.024+0000 7fd9abfff640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ac160370 0x7fd9ac0c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.025+0000 7fd9aaffd640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9ac15f8a0 0x7fd9ac1344d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd9a002a270 tx=0x7fd9a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.025+0000 7fd993fff640 1 -- 172.21.15.37:0/2567411293 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0037040 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.025+0000 7fd993fff640 1 -- 172.21.15.37:0/2567411293 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a0042070 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.025+0000 7fd9b2020640 1 -- 172.21.15.37:0/2567411293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ac15a8b0 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.025+0000 7fd993fff640 1 -- 172.21.15.37:0/2567411293 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0030bb0 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.025+0000 7fd9b2020640 1 -- 172.21.15.37:0/2567411293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ac15ae70 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.026+0000 7fd9b2020640 1 -- 172.21.15.37:0/2567411293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9ac06a360 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.026+0000 7fd993fff640 1 -- 172.21.15.37:0/2567411293 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fd9a0047460 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.027+0000 7fd993fff640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd99806cbc0 0x7fd99806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.027+0000 7fd993fff640 1 -- 172.21.15.37:0/2567411293 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7fd9a00acba0 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.027+0000 7fd9ab7fe640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd99806cbc0 0x7fd99806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.027+0000 7fd9ab7fe640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd99806cbc0 0x7fd99806f080 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd99c007fd0 tx=0x7fd99c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.031+0000 7fd993fff640 1 -- 172.21.15.37:0/2567411293 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd9ac0b4eb0 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.217+0000 7fd9b2020640 1 -- 172.21.15.37:0/2567411293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd9ac069480 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.218+0000 7fd993fff640 1 -- 172.21.15.37:0/2567411293 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7fd9a0045360 con 0x7fd9ac15f8a0 2024-04-26T20:23:53.219 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:23:53.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 -- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd99806cbc0 msgr2=0x7fd99806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd99806cbc0 0x7fd99806f080 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd99c007fd0 tx=0x7fd99c009040 comp rx=0 tx=0).stop 2024-04-26T20:23:53.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 -- 172.21.15.37:0/2567411293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9ac15f8a0 msgr2=0x7fd9ac1344d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9ac15f8a0 0x7fd9ac1344d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd9a002a270 tx=0x7fd9a00046c0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 -- 172.21.15.37:0/2567411293 shutdown_connections 2024-04-26T20:23:53.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ac160370 0x7fd9ac0c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9ac15f8a0 0x7fd9ac1344d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd99806cbc0 0x7fd99806f080 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 --2- 172.21.15.37:0/2567411293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9ac0b4aa0 0x7fd9ac159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 -- 172.21.15.37:0/2567411293 >> 172.21.15.37:0/2567411293 conn(0x7fd9ac0d16b0 msgr2=0x7fd9ac075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:53.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 -- 172.21.15.37:0/2567411293 shutdown_connections 2024-04-26T20:23:53.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.220+0000 7fd9b2020640 1 -- 172.21.15.37:0/2567411293 wait complete. 2024-04-26T20:23:53.232 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:23:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.332+0000 7f3b1d8c5640 1 Processor -- start 2024-04-26T20:23:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.333+0000 7f3b1d8c5640 1 -- start start 2024-04-26T20:23:53.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.333+0000 7f3b1d8c5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b1806f790 0x7f3b18074030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.333+0000 7f3b1d8c5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3b180b6b10 0x7f3b181611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.333+0000 7f3b1d8c5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b18161b80 con 0x7f3b18160d70 2024-04-26T20:23:53.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.333+0000 7f3b1d8c5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3b18161710 con 0x7f3b180b6b10 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.333+0000 7f3b1d8c5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3b180b4400 con 0x7f3b1806f790 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.334+0000 7f3b16ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b1806f790 0x7f3b18074030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.334+0000 7f3b177fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3b180b6b10 0x7f3b181611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.334+0000 7f3b16ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b1806f790 0x7f3b18074030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:37398/0 (socket says 172.21.15.37:37398) 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.334+0000 7f3b16ffd640 1 -- 172.21.15.37:0/2462954494 learned_addr learned my addr 172.21.15.37:0/2462954494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.334+0000 7f3b167fc640 1 --1- 172.21.15.37:0/2462954494 >> v1:172.21.15.134:6789/0 conn(0x7f3b18160d70 0x7f3b1815d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:60514/0 (socket says 172.21.15.37:60514) 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.334+0000 7f3b15ffb640 1 -- 172.21.15.37:0/2462954494 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3359766725 0 0) 0x7f3b18161b80 con 0x7f3b18160d70 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.334+0000 7f3b15ffb640 1 -- 172.21.15.37:0/2462954494 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b04003520 con 0x7f3b18160d70 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.334+0000 7f3b177fe640 1 -- 172.21.15.37:0/2462954494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b1806f790 msgr2=0x7f3b18074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.334+0000 7f3b177fe640 1 --2- 172.21.15.37:0/2462954494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b1806f790 0x7f3b18074030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.334+0000 7f3b177fe640 1 -- 172.21.15.37:0/2462954494 >> v1:172.21.15.134:6789/0 conn(0x7f3b18160d70 legacy=0x7f3b1815d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.334+0000 7f3b177fe640 1 -- 172.21.15.37:0/2462954494 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b18074570 con 0x7f3b180b6b10 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.335+0000 7f3b177fe640 1 --2- 172.21.15.37:0/2462954494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b180b6b10 0x7f3b181611d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009d90 tx=0x7f3b1002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20d099f03b12f455 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.335+0000 7f3b15ffb640 1 -- 172.21.15.37:0/2462954494 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10007740 con 0x7f3b180b6b10 2024-04-26T20:23:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.335+0000 7f3b15ffb640 1 -- 172.21.15.37:0/2462954494 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b100078e0 con 0x7f3b180b6b10 2024-04-26T20:23:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.335+0000 7f3b15ffb640 1 -- 172.21.15.37:0/2462954494 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10007c90 con 0x7f3b180b6b10 2024-04-26T20:23:53.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.336+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/2462954494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b180b6b10 msgr2=0x7f3b181611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.336+0000 7f3b1d8c5640 1 --2- 172.21.15.37:0/2462954494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b180b6b10 0x7f3b181611d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009d90 tx=0x7f3b1002a660 comp rx=0 tx=0).stop 2024-04-26T20:23:53.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.336+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/2462954494 shutdown_connections 2024-04-26T20:23:53.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.336+0000 7f3b1d8c5640 1 --2- 172.21.15.37:0/2462954494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b180b6b10 0x7f3b181611d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.336+0000 7f3b1d8c5640 1 --2- 172.21.15.37:0/2462954494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b1806f790 0x7f3b18074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.336+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/2462954494 >> 172.21.15.37:0/2462954494 conn(0x7f3b180d0d50 msgr2=0x7f3b180d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.336+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/2462954494 shutdown_connections 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.336+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/2462954494 wait complete. 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.337+0000 7f3b1d8c5640 1 Processor -- start 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.337+0000 7f3b1d8c5640 1 -- start start 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b1d8c5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b1806f790 0x7f3b1813c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b1d8c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b180b6b10 0x7f3b181426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b1d8c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b18160d70 0x7f3b1813d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b1d8c5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3b180732d0 con 0x7f3b18160d70 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b1d8c5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18072ff0 con 0x7f3b180b6b10 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b1d8c5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3b180735b0 con 0x7f3b1806f790 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b18160d70 0x7f3b1813d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b18160d70 0x7f3b1813d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37168/0 (socket says 172.21.15.37:37168) 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b177fe640 1 -- 172.21.15.37:0/3822150463 learned_addr learned my addr 172.21.15.37:0/3822150463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b167fc640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b180b6b10 0x7f3b181426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b16ffd640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b1806f790 0x7f3b1813c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b177fe640 1 -- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b1806f790 msgr2=0x7f3b1813c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b177fe640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b1806f790 0x7f3b1813c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b177fe640 1 -- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b180b6b10 msgr2=0x7f3b181426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b177fe640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b180b6b10 0x7f3b181426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.338+0000 7f3b177fe640 1 -- 172.21.15.37:0/3822150463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1813ef40 con 0x7f3b18160d70 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.339+0000 7f3b177fe640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b18160d70 0x7f3b1813d9d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009eb0 tx=0x7f3b10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.339+0000 7f3b0ffff640 1 -- 172.21.15.37:0/3822150463 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1003f070 con 0x7f3b18160d70 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.339+0000 7f3b0ffff640 1 -- 172.21.15.37:0/3822150463 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b10031c60 con 0x7f3b18160d70 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.339+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/3822150463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1813e6a0 con 0x7f3b18160d70 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.339+0000 7f3b0ffff640 1 -- 172.21.15.37:0/3822150463 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10009450 con 0x7f3b18160d70 2024-04-26T20:23:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.339+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/3822150463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1813ec60 con 0x7f3b18160d70 2024-04-26T20:23:53.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.340+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/3822150463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b1806a360 con 0x7f3b18160d70 2024-04-26T20:23:53.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.340+0000 7f3b0ffff640 1 -- 172.21.15.37:0/3822150463 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f3b1003b410 con 0x7f3b18160d70 2024-04-26T20:23:53.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.341+0000 7f3b0ffff640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3af806cbf0 0x7f3af806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.341+0000 7f3b0ffff640 1 -- 172.21.15.37:0/3822150463 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f3b100ad2e0 con 0x7f3b18160d70 2024-04-26T20:23:53.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.341+0000 7f3b16ffd640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3af806cbf0 0x7f3af806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.341+0000 7f3b16ffd640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3af806cbf0 0x7f3af806f0b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009b30 tx=0x7f3b00009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.345+0000 7f3b0ffff640 1 -- 172.21.15.37:0/3822150463 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b1007ac40 con 0x7f3b18160d70 2024-04-26T20:23:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.544+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/3822150463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3b1806ff40 con 0x7f3b18160d70 2024-04-26T20:23:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.545+0000 7f3b0ffff640 1 -- 172.21.15.37:0/3822150463 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7f3b1007ef40 con 0x7f3b18160d70 2024-04-26T20:23:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-26T20:23:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3af806cbf0 msgr2=0x7f3af806f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3af806cbf0 0x7f3af806f0b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009b30 tx=0x7f3b00009240 comp rx=0 tx=0).stop 2024-04-26T20:23:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/3822150463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b18160d70 msgr2=0x7f3b1813d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b18160d70 0x7f3b1813d9d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009eb0 tx=0x7f3b10004a40 comp rx=0 tx=0).stop 2024-04-26T20:23:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/3822150463 shutdown_connections 2024-04-26T20:23:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b18160d70 0x7f3b1813d9d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b180b6b10 0x7f3b181426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3af806cbf0 0x7f3af806f0b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 --2- 172.21.15.37:0/3822150463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b1806f790 0x7f3b1813c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/3822150463 >> 172.21.15.37:0/3822150463 conn(0x7f3b180d0d50 msgr2=0x7f3b180cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:53.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/3822150463 shutdown_connections 2024-04-26T20:23:53.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.547+0000 7f3b1d8c5640 1 -- 172.21.15.37:0/3822150463 wait complete. 2024-04-26T20:23:53.559 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T20:23:53.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:53.559 DEBUG:teuthology.orchestra.run.smithi134:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T20:23:53.590 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:23:53.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:23:53.591 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T20:23:53.648 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:23:53.648 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:23:53.649 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:53.664 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=371 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:23:53.664 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:23:53.665 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:23:53.665 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:23:53.705 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17743 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:23:53.706 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:23:53.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.806+0000 7f0027ad1640 1 Processor -- start 2024-04-26T20:23:53.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.806+0000 7f0027ad1640 1 -- start start 2024-04-26T20:23:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.807+0000 7f0027ad1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f00200cf810 0x7f002016a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.807+0000 7f0027ad1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00200d2020 con 0x7f002006e530 2024-04-26T20:23:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.807+0000 7f0027ad1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00200ce700 con 0x7f00200d1c40 2024-04-26T20:23:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.807+0000 7f0027ad1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f00200ce4b0 con 0x7f00200cf810 2024-04-26T20:23:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.807+0000 7f002506c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f002006e530 0x7f00200cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53866/0 (socket says 172.21.15.37:53866) 2024-04-26T20:23:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.807+0000 7f002506c640 1 -- 172.21.15.37:0/758024207 learned_addr learned my addr 172.21.15.37:0/758024207 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.807+0000 7f002606e640 1 --2- 172.21.15.37:0/758024207 >> v2:172.21.15.134:3300/0 conn(0x7f00200cf810 0x7f002016a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.807+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220375643 0 0) 0x7f00200d2020 con 0x7f002006e530 2024-04-26T20:23:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.807+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f000c003520 con 0x7f002006e530 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3090304146 0 0) 0x7f000c003520 con 0x7f002006e530 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 >> v2:172.21.15.134:3300/0 conn(0x7f00200cf810 msgr2=0x7f002016a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 --2- 172.21.15.37:0/758024207 >> v2:172.21.15.134:3300/0 conn(0x7f00200cf810 0x7f002016a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 >> v1:172.21.15.134:6789/0 conn(0x7f00200d1c40 legacy=0x7f002006fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00200ce9e0 con 0x7f002006e530 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f0010001670 con 0x7f002006e530 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 >> v1:172.21.15.37:6789/0 conn(0x7f002006e530 legacy=0x7f00200cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 --2- 172.21.15.37:0/758024207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f000c0039b0 0x7f000c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 --2- 172.21.15.37:0/758024207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c004800 0x7f000c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 --2- 172.21.15.37:0/758024207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f000c0051c0 0x7f000c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00200ce4b0 con 0x7f000c0039b0 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f000c00c060 con 0x7f000c004800 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f000c00c260 con 0x7f000c0051c0 2024-04-26T20:23:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002586d640 1 --2- 172.21.15.37:0/758024207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f000c0039b0 0x7f000c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002506c640 1 --2- 172.21.15.37:0/758024207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c004800 0x7f000c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002606e640 1 --2- 172.21.15.37:0/758024207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f000c0051c0 0x7f000c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002586d640 1 -- 172.21.15.37:0/758024207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f000c0051c0 msgr2=0x7f000c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002586d640 1 --2- 172.21.15.37:0/758024207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f000c0051c0 0x7f000c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002586d640 1 -- 172.21.15.37:0/758024207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c004800 msgr2=0x7f000c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002586d640 1 --2- 172.21.15.37:0/758024207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c004800 0x7f000c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002586d640 1 -- 172.21.15.37:0/758024207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c003520 con 0x7f000c0039b0 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.808+0000 7f002606e640 1 -- 172.21.15.37:0/758024207 reap_dead start 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.809+0000 7f002586d640 1 --2- 172.21.15.37:0/758024207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f000c0039b0 0x7f000c003dd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f001c001750 tx=0x7f000c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c3718b7b9a9ead server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.809+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c001370 con 0x7f000c0039b0 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.809+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f001c0052d0 con 0x7f000c0039b0 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.809+0000 7f002486b640 1 -- 172.21.15.37:0/758024207 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c02abe0 con 0x7f000c0039b0 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.809+0000 7f0027ad1640 1 -- 172.21.15.37:0/758024207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f000c0039b0 msgr2=0x7f000c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.809+0000 7f0027ad1640 1 --2- 172.21.15.37:0/758024207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f000c0039b0 0x7f000c003dd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f001c001750 tx=0x7f000c005830 comp rx=0 tx=0).stop 2024-04-26T20:23:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.809+0000 7f0027ad1640 1 -- 172.21.15.37:0/758024207 shutdown_connections 2024-04-26T20:23:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.809+0000 7f0027ad1640 1 --2- 172.21.15.37:0/758024207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f000c0039b0 0x7f000c003dd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.809+0000 7f0027ad1640 1 -- 172.21.15.37:0/758024207 >> 172.21.15.37:0/758024207 conn(0x7f0020073880 msgr2=0x7f0020073c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.810+0000 7f0027ad1640 1 -- 172.21.15.37:0/758024207 shutdown_connections 2024-04-26T20:23:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.810+0000 7f0027ad1640 1 -- 172.21.15.37:0/758024207 wait complete. 2024-04-26T20:23:53.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.811+0000 7f0027ad1640 1 Processor -- start 2024-04-26T20:23:53.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.811+0000 7f0027ad1640 1 -- start start 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.811+0000 7f0027ad1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c0039b0 0x7f0020155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.811+0000 7f0027ad1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00200aeee0 0x7f0020157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.811+0000 7f0027ad1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0020157ed0 0x7f00201f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f0027ad1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00200cbb30 con 0x7f00200aeee0 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f0027ad1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00200cb850 con 0x7f000c0039b0 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f0027ad1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f00200cf100 con 0x7f0020157ed0 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002506c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00200aeee0 0x7f0020157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002506c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00200aeee0 0x7f0020157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37190/0 (socket says 172.21.15.37:37190) 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002506c640 1 -- 172.21.15.37:0/1300980689 learned_addr learned my addr 172.21.15.37:0/1300980689 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002586d640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c0039b0 0x7f0020155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002606e640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0020157ed0 0x7f00201f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002506c640 1 -- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0020157ed0 msgr2=0x7f00201f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002506c640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0020157ed0 0x7f00201f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002506c640 1 -- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c0039b0 msgr2=0x7f0020155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002506c640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c0039b0 0x7f0020155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002506c640 1 -- 172.21.15.37:0/1300980689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0020155580 con 0x7f00200aeee0 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002586d640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c0039b0 0x7f0020155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f002506c640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00200aeee0 0x7f0020157990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f001000e760 tx=0x7f001000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f00167fc640 1 -- 172.21.15.37:0/1300980689 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001000c660 con 0x7f00200aeee0 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f00167fc640 1 -- 172.21.15.37:0/1300980689 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0010018070 con 0x7f00200aeee0 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f0027ad1640 1 -- 172.21.15.37:0/1300980689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0020155da0 con 0x7f00200aeee0 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f00167fc640 1 -- 172.21.15.37:0/1300980689 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001000ca20 con 0x7f00200aeee0 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.812+0000 7f0027ad1640 1 -- 172.21.15.37:0/1300980689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0020156360 con 0x7f00200aeee0 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.813+0000 7f0027ad1640 1 -- 172.21.15.37:0/1300980689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f002006a360 con 0x7f00200aeee0 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.814+0000 7f00167fc640 1 -- 172.21.15.37:0/1300980689 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f001000cbc0 con 0x7f00200aeee0 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.814+0000 7f00167fc640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f000406cbf0 0x7f000406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.815+0000 7f00167fc640 1 -- 172.21.15.37:0/1300980689 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f001008ba40 con 0x7f00200aeee0 2024-04-26T20:23:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.815+0000 7f002586d640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f000406cbf0 0x7f000406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.815+0000 7f002586d640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f000406cbf0 0x7f000406f0b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f001c032000 tx=0x7f001c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:53.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:53.819+0000 7f00167fc640 1 -- 172.21.15.37:0/1300980689 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0010059a70 con 0x7f00200aeee0 2024-04-26T20:23:54.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.049+0000 7f0027ad1640 1 -- 172.21.15.37:0/1300980689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00200d1650 con 0x7f00200aeee0 2024-04-26T20:23:54.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.049+0000 7f00167fc640 1 -- 172.21.15.37:0/1300980689 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5181 (secure 0 0 0) 0x7f0010059370 con 0x7f00200aeee0 2024-04-26T20:23:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:23:54.051 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/4166977617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4166977617},{"type":"v1","addr":"172.21.15.134:6836","nonce":4166977617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3245561282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3245561282},{"type":"v1","addr":"172.21.15.134:6837","nonce":3245561282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3853963851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3853963851},{"type":"v1","addr":"172.21.15.37:6835","nonce":3853963851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:22:42.116910+0000","modified":"2024-04-26T20:22:45.119356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5097":{"gid":5097,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/64661056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":64661056},{"type":"v1","addr":"172.21.15.37:6834","nonce":64661056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5097,"qdb_cluster":[5097]},"id":2}]} 2024-04-26T20:23:54.051 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 16 2024-04-26T20:23:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 -- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f000406cbf0 msgr2=0x7f000406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f000406cbf0 0x7f000406f0b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f001c032000 tx=0x7f001c02e000 comp rx=0 tx=0).stop 2024-04-26T20:23:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 -- 172.21.15.37:0/1300980689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00200aeee0 msgr2=0x7f0020157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00200aeee0 0x7f0020157990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f001000e760 tx=0x7f001000ec20 comp rx=0 tx=0).stop 2024-04-26T20:23:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 -- 172.21.15.37:0/1300980689 shutdown_connections 2024-04-26T20:23:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f000406cbf0 0x7f000406f0b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0020157ed0 0x7f00201f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00200aeee0 0x7f0020157990 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 --2- 172.21.15.37:0/1300980689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c0039b0 0x7f0020155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 -- 172.21.15.37:0/1300980689 >> 172.21.15.37:0/1300980689 conn(0x7f0020073880 msgr2=0x7f0020073c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 -- 172.21.15.37:0/1300980689 shutdown_connections 2024-04-26T20:23:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.052+0000 7f0027ad1640 1 -- 172.21.15.37:0/1300980689 wait complete. 2024-04-26T20:23:54.064 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:23:54.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.164+0000 7f7ea9be4640 1 Processor -- start 2024-04-26T20:23:54.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.164+0000 7f7ea9be4640 1 -- start start 2024-04-26T20:23:54.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.164+0000 7f7ea9be4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7ea415f380 0x7f7ea40d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.164+0000 7f7ea9be4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ea40b9a90 0x7f7ea40d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.165+0000 7f7ea9be4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea40d3de0 con 0x7f7ea415f760 2024-04-26T20:23:54.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.165+0000 7f7ea9be4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea40d43a0 con 0x7f7ea415f380 2024-04-26T20:23:54.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.165+0000 7f7ea9be4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea40d40c0 con 0x7f7ea40b9a90 2024-04-26T20:23:54.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.165+0000 7f7ea37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7ea415f380 0x7f7ea40d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.165+0000 7f7ea37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7ea415f380 0x7f7ea40d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37206/0 (socket says 172.21.15.37:37206) 2024-04-26T20:23:54.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.165+0000 7f7ea37fe640 1 -- 172.21.15.37:0/1308594368 learned_addr learned my addr 172.21.15.37:0/1308594368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:54.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.165+0000 7f7ea3fff640 1 --2- 172.21.15.37:0/1308594368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ea40b9a90 0x7f7ea40d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.165+0000 7f7ea37fe640 1 -- 172.21.15.37:0/1308594368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ea40b9a90 msgr2=0x7f7ea40d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:54.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.165+0000 7f7ea37fe640 1 --2- 172.21.15.37:0/1308594368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ea40b9a90 0x7f7ea40d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.165+0000 7f7ea37fe640 1 -- 172.21.15.37:0/1308594368 >> v1:172.21.15.134:6789/0 conn(0x7f7ea415f760 legacy=0x7f7ea40cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:54.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.165+0000 7f7ea37fe640 1 -- 172.21.15.37:0/1308594368 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea40b8280 con 0x7f7ea415f380 2024-04-26T20:23:54.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea37fe640 1 --2- 172.21.15.37:0/1308594368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea415f380 0x7f7ea40d1bd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009870 tx=0x7f7e9402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72e3a33db28dc3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:54.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea27fc640 1 -- 172.21.15.37:0/1308594368 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94037070 con 0x7f7ea415f380 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea27fc640 1 -- 172.21.15.37:0/1308594368 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e94032440 con 0x7f7ea415f380 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea27fc640 1 -- 172.21.15.37:0/1308594368 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94032810 con 0x7f7ea415f380 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea9be4640 1 -- 172.21.15.37:0/1308594368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea415f380 msgr2=0x7f7ea40d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea9be4640 1 --2- 172.21.15.37:0/1308594368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea415f380 0x7f7ea40d1bd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009870 tx=0x7f7e9402a2a0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea9be4640 1 -- 172.21.15.37:0/1308594368 shutdown_connections 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea9be4640 1 --2- 172.21.15.37:0/1308594368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ea40b9a90 0x7f7ea40d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea9be4640 1 --2- 172.21.15.37:0/1308594368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea415f380 0x7f7ea40d1bd0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea9be4640 1 -- 172.21.15.37:0/1308594368 >> 172.21.15.37:0/1308594368 conn(0x7f7ea409e560 msgr2=0x7f7ea40d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea9be4640 1 -- 172.21.15.37:0/1308594368 shutdown_connections 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.166+0000 7f7ea9be4640 1 -- 172.21.15.37:0/1308594368 wait complete. 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.167+0000 7f7ea9be4640 1 Processor -- start 2024-04-26T20:23:54.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.168+0000 7f7ea9be4640 1 -- start start 2024-04-26T20:23:54.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.168+0000 7f7ea9be4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ea40b9a90 0x7f7ea4152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.168+0000 7f7ea9be4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ea415f380 0x7f7ea40cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.168+0000 7f7ea9be4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea415f760 0x7f7ea415d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.168+0000 7f7ea9be4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40d1280 con 0x7f7ea415f760 2024-04-26T20:23:54.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.168+0000 7f7ea9be4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40d1100 con 0x7f7ea40b9a90 2024-04-26T20:23:54.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.168+0000 7f7ea9be4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea40d5b50 con 0x7f7ea415f380 2024-04-26T20:23:54.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.168+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea415f760 0x7f7ea415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea415f760 0x7f7ea415d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37222/0 (socket says 172.21.15.37:37222) 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea3fff640 1 -- 172.21.15.37:0/3840623993 learned_addr learned my addr 172.21.15.37:0/3840623993 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea2ffd640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ea415f380 0x7f7ea40cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea37fe640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ea40b9a90 0x7f7ea4152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea3fff640 1 -- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ea415f380 msgr2=0x7f7ea40cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea3fff640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ea415f380 0x7f7ea40cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea3fff640 1 -- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ea40b9a90 msgr2=0x7f7ea4152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea3fff640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ea40b9a90 0x7f7ea4152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea3fff640 1 -- 172.21.15.37:0/3840623993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea40cc120 con 0x7f7ea415f760 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea3fff640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea415f760 0x7f7ea415d100 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c007e60 tx=0x7f7e9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7e8bfff640 1 -- 172.21.15.37:0/3840623993 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c014620 con 0x7f7ea415f760 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7e8bfff640 1 -- 172.21.15.37:0/3840623993 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c019070 con 0x7f7ea415f760 2024-04-26T20:23:54.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea9be4640 1 -- 172.21.15.37:0/3840623993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea41f53a0 con 0x7f7ea415f760 2024-04-26T20:23:54.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7e8bfff640 1 -- 172.21.15.37:0/3840623993 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c014a00 con 0x7f7ea415f760 2024-04-26T20:23:54.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.169+0000 7f7ea9be4640 1 -- 172.21.15.37:0/3840623993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea41f5960 con 0x7f7ea415f760 2024-04-26T20:23:54.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.170+0000 7f7ea9be4640 1 -- 172.21.15.37:0/3840623993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea406a360 con 0x7f7ea415f760 2024-04-26T20:23:54.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.171+0000 7f7e8bfff640 1 -- 172.21.15.37:0/3840623993 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f7e9c014ba0 con 0x7f7ea415f760 2024-04-26T20:23:54.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.171+0000 7f7e8bfff640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7e9006cbc0 0x7f7e9006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.171+0000 7f7e8bfff640 1 -- 172.21.15.37:0/3840623993 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f7e9c08c920 con 0x7f7ea415f760 2024-04-26T20:23:54.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.171+0000 7f7ea37fe640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7e9006cbc0 0x7f7e9006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.172+0000 7f7ea37fe640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7e9006cbc0 0x7f7e9006f080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7e94034000 tx=0x7f7e9402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.175+0000 7f7e8bfff640 1 -- 172.21.15.37:0/3840623993 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e9c05a9e0 con 0x7f7ea415f760 2024-04-26T20:23:54.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.364+0000 7f7ea9be4640 1 -- 172.21.15.37:0/3840623993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ea415edc0 con 0x7f7ea415f760 2024-04-26T20:23:54.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.366+0000 7f7e8bfff640 1 -- 172.21.15.37:0/3840623993 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16746 (secure 0 0 0) 0x7f7e9c05a2e0 con 0x7f7ea415f760 2024-04-26T20:23:54.367 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:23:54.367 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:23:34.988080+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:22:39.762703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:22:41.066360+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:23:54.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 -- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7e9006cbc0 msgr2=0x7f7e9006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7e9006cbc0 0x7f7e9006f080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7e94034000 tx=0x7f7e9402e000 comp rx=0 tx=0).stop 2024-04-26T20:23:54.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 -- 172.21.15.37:0/3840623993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea415f760 msgr2=0x7f7ea415d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea415f760 0x7f7ea415d100 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c007e60 tx=0x7f7e9c00e4b0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 -- 172.21.15.37:0/3840623993 shutdown_connections 2024-04-26T20:23:54.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ea415f760 0x7f7ea415d100 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7e9006cbc0 0x7f7e9006f080 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ea415f380 0x7f7ea40cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 --2- 172.21.15.37:0/3840623993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ea40b9a90 0x7f7ea4152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 -- 172.21.15.37:0/3840623993 >> 172.21.15.37:0/3840623993 conn(0x7f7ea409e560 msgr2=0x7f7ea406cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:54.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 -- 172.21.15.37:0/3840623993 shutdown_connections 2024-04-26T20:23:54.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.368+0000 7f7ea9be4640 1 -- 172.21.15.37:0/3840623993 wait complete. 2024-04-26T20:23:54.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:22:39.762703+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:22:41.066360+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:23:54.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:23:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.483+0000 7f4780f38640 1 Processor -- start 2024-04-26T20:23:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.483+0000 7f4780f38640 1 -- start start 2024-04-26T20:23:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4780f38640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f477c0cff80 0x7f477c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4780f38640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f477c0d1fe0 0x7f477c0d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4780f38640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f477c0cd640 con 0x7f477c0b2290 2024-04-26T20:23:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4780f38640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f477c0cd920 con 0x7f477c0cff80 2024-04-26T20:23:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4780f38640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f477c0cdc00 con 0x7f477c0d1fe0 2024-04-26T20:23:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4773fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f477c0cff80 0x7f477c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4773fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f477c0cff80 0x7f477c0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37228/0 (socket says 172.21.15.37:37228) 2024-04-26T20:23:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4773fff640 1 -- 172.21.15.37:0/1961229836 learned_addr learned my addr 172.21.15.37:0/1961229836 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f477ad9d640 1 --2- 172.21.15.37:0/1961229836 >> v2:172.21.15.134:3300/0 conn(0x7f477c0d1fe0 0x7f477c0d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4773fff640 1 -- 172.21.15.37:0/1961229836 >> v2:172.21.15.134:3300/0 conn(0x7f477c0d1fe0 msgr2=0x7f477c0d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4773fff640 1 --2- 172.21.15.37:0/1961229836 >> v2:172.21.15.134:3300/0 conn(0x7f477c0d1fe0 0x7f477c0d23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4773fff640 1 -- 172.21.15.37:0/1961229836 >> v1:172.21.15.134:6789/0 conn(0x7f477c0b2290 legacy=0x7f477c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f4773fff640 1 -- 172.21.15.37:0/1961229836 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c0cdee0 con 0x7f477c0cff80 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.484+0000 7f477ad9d640 1 --2- 172.21.15.37:0/1961229836 >> v2:172.21.15.134:3300/0 conn(0x7f477c0d1fe0 0x7f477c0d23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.485+0000 7f4773fff640 1 --2- 172.21.15.37:0/1961229836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f477c0cff80 0x7f477c0b2930 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f477c074160 tx=0x7f476402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429edb0d8c048556 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.485+0000 7f4779d9b640 1 -- 172.21.15.37:0/1961229836 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764037070 con 0x7f477c0cff80 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.485+0000 7f4779d9b640 1 -- 172.21.15.37:0/1961229836 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4764032440 con 0x7f477c0cff80 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.485+0000 7f4779d9b640 1 -- 172.21.15.37:0/1961229836 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764032810 con 0x7f477c0cff80 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.485+0000 7f4780f38640 1 -- 172.21.15.37:0/1961229836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f477c0cff80 msgr2=0x7f477c0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.485+0000 7f4780f38640 1 --2- 172.21.15.37:0/1961229836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f477c0cff80 0x7f477c0b2930 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f477c074160 tx=0x7f476402a230 comp rx=0 tx=0).stop 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.485+0000 7f4780f38640 1 -- 172.21.15.37:0/1961229836 shutdown_connections 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.485+0000 7f4780f38640 1 --2- 172.21.15.37:0/1961229836 >> v2:172.21.15.134:3300/0 conn(0x7f477c0d1fe0 0x7f477c0d23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.485+0000 7f4780f38640 1 --2- 172.21.15.37:0/1961229836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f477c0cff80 0x7f477c0b2930 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.485+0000 7f4780f38640 1 -- 172.21.15.37:0/1961229836 >> 172.21.15.37:0/1961229836 conn(0x7f477c0d2ff0 msgr2=0x7f477c0d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:54.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.485+0000 7f4780f38640 1 -- 172.21.15.37:0/1961229836 shutdown_connections 2024-04-26T20:23:54.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.486+0000 7f4780f38640 1 -- 172.21.15.37:0/1961229836 wait complete. 2024-04-26T20:23:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.487+0000 7f4780f38640 1 Processor -- start 2024-04-26T20:23:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.487+0000 7f4780f38640 1 -- start start 2024-04-26T20:23:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.487+0000 7f4780f38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f477c0b2290 0x7f477c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.487+0000 7f4780f38640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f477c0cff80 0x7f477c160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.487+0000 7f4780f38640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f477c0d1fe0 0x7f477c161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.487+0000 7f4780f38640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f477c0d2930 con 0x7f477c0d1fe0 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.487+0000 7f4780f38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f477c0b2670 con 0x7f477c0b2290 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.487+0000 7f4780f38640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f477c0d2ab0 con 0x7f477c0cff80 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f477ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f477c0d1fe0 0x7f477c161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f477ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f477c0d1fe0 0x7f477c161400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37236/0 (socket says 172.21.15.37:37236) 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f477ad9d640 1 -- 172.21.15.37:0/2271479343 learned_addr learned my addr 172.21.15.37:0/2271479343 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f4773fff640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f477c0cff80 0x7f477c160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f477a59c640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f477c0b2290 0x7f477c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f477ad9d640 1 -- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f477c0cff80 msgr2=0x7f477c160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f477ad9d640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f477c0cff80 0x7f477c160ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f477ad9d640 1 -- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f477c0b2290 msgr2=0x7f477c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f477ad9d640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f477c0b2290 0x7f477c15f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f477ad9d640 1 -- 172.21.15.37:0/2271479343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c0ccc40 con 0x7f477c0d1fe0 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f477ad9d640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f477c0d1fe0 0x7f477c161400 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f477400cd50 tx=0x7f477400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f4772ffd640 1 -- 172.21.15.37:0/2271479343 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477400f680 con 0x7f477c0d1fe0 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f4772ffd640 1 -- 172.21.15.37:0/2271479343 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477400f840 con 0x7f477c0d1fe0 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f4780f38640 1 -- 172.21.15.37:0/2271479343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477c0b5fe0 con 0x7f477c0d1fe0 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f4772ffd640 1 -- 172.21.15.37:0/2271479343 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477400fc30 con 0x7f477c0d1fe0 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.488+0000 7f4780f38640 1 -- 172.21.15.37:0/2271479343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477c0b65a0 con 0x7f477c0d1fe0 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.489+0000 7f4780f38640 1 -- 172.21.15.37:0/2271479343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f477c06a360 con 0x7f477c0d1fe0 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.490+0000 7f4772ffd640 1 -- 172.21.15.37:0/2271479343 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f47740213f0 con 0x7f477c0d1fe0 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.490+0000 7f4772ffd640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f475c06cbc0 0x7f475c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.490+0000 7f4772ffd640 1 -- 172.21.15.37:0/2271479343 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f477408cfe0 con 0x7f477c0d1fe0 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.491+0000 7f477a59c640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f475c06cbc0 0x7f475c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.491+0000 7f477a59c640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f475c06cbc0 0x7f475c06f080 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f476c003d90 tx=0x7f476c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:54.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.495+0000 7f4772ffd640 1 -- 172.21.15.37:0/2271479343 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4774014030 con 0x7f477c0d1fe0 2024-04-26T20:23:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.678+0000 7f4780f38640 1 -- 172.21.15.37:0/2271479343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f477c076100 con 0x7f477c0d1fe0 2024-04-26T20:23:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.679+0000 7f4772ffd640 1 -- 172.21.15.37:0/2271479343 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f477405aeb0 con 0x7f477c0d1fe0 2024-04-26T20:23:54.680 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:23:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 -- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f475c06cbc0 msgr2=0x7f475c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f475c06cbc0 0x7f475c06f080 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f476c003d90 tx=0x7f476c009040 comp rx=0 tx=0).stop 2024-04-26T20:23:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 -- 172.21.15.37:0/2271479343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f477c0d1fe0 msgr2=0x7f477c161400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f477c0d1fe0 0x7f477c161400 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f477400cd50 tx=0x7f477400a650 comp rx=0 tx=0).stop 2024-04-26T20:23:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 -- 172.21.15.37:0/2271479343 shutdown_connections 2024-04-26T20:23:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f477c0d1fe0 0x7f477c161400 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f475c06cbc0 0x7f475c06f080 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f477c0cff80 0x7f477c160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 --2- 172.21.15.37:0/2271479343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f477c0b2290 0x7f477c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 -- 172.21.15.37:0/2271479343 >> 172.21.15.37:0/2271479343 conn(0x7f477c0d2ff0 msgr2=0x7f477c0b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 -- 172.21.15.37:0/2271479343 shutdown_connections 2024-04-26T20:23:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.681+0000 7f4780f38640 1 -- 172.21.15.37:0/2271479343 wait complete. 2024-04-26T20:23:54.694 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:23:54.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.798+0000 7f300187b640 1 Processor -- start 2024-04-26T20:23:54.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.798+0000 7f300187b640 1 -- start start 2024-04-26T20:23:54.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.799+0000 7f300187b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2ffc15d320 0x7f2ffc0d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.799+0000 7f300187b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2ffc0d1090 0x7f2ffc0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.799+0000 7f300187b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ffc15d700 con 0x7f2ffc0cf990 2024-04-26T20:23:54.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.799+0000 7f300187b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2ffc0ceeb0 con 0x7f2ffc15d320 2024-04-26T20:23:54.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.799+0000 7f300187b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2ffc0ce8f0 con 0x7f2ffc0d1090 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.799+0000 7f2ffaffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2ffc15d320 0x7f2ffc0d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.799+0000 7f2ffaffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2ffc15d320 0x7f2ffc0d0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37246/0 (socket says 172.21.15.37:37246) 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.799+0000 7f2ffaffd640 1 -- 172.21.15.37:0/2195615755 learned_addr learned my addr 172.21.15.37:0/2195615755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.799+0000 7f2ffa7fc640 1 --2- 172.21.15.37:0/2195615755 >> v2:172.21.15.134:3300/0 conn(0x7f2ffc0d1090 0x7f2ffc0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.800+0000 7f2ffaffd640 1 -- 172.21.15.37:0/2195615755 >> v2:172.21.15.134:3300/0 conn(0x7f2ffc0d1090 msgr2=0x7f2ffc0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.800+0000 7f2ffaffd640 1 --2- 172.21.15.37:0/2195615755 >> v2:172.21.15.134:3300/0 conn(0x7f2ffc0d1090 0x7f2ffc0d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.800+0000 7f2ffaffd640 1 -- 172.21.15.37:0/2195615755 >> v1:172.21.15.134:6789/0 conn(0x7f2ffc0cf990 legacy=0x7f2ffc0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.800+0000 7f2ffaffd640 1 -- 172.21.15.37:0/2195615755 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc0cebd0 con 0x7f2ffc15d320 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.800+0000 7f2ffaffd640 1 --2- 172.21.15.37:0/2195615755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ffc15d320 0x7f2ffc0d0b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009870 tx=0x7f2ff402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8f122ef5ab0943 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.801+0000 7f2ff9ffb640 1 -- 172.21.15.37:0/2195615755 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4037070 con 0x7f2ffc15d320 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.801+0000 7f2ff9ffb640 1 -- 172.21.15.37:0/2195615755 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff4032440 con 0x7f2ffc15d320 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.801+0000 7f2ff9ffb640 1 -- 172.21.15.37:0/2195615755 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4032810 con 0x7f2ffc15d320 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.801+0000 7f300187b640 1 -- 172.21.15.37:0/2195615755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ffc15d320 msgr2=0x7f2ffc0d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.801+0000 7f300187b640 1 --2- 172.21.15.37:0/2195615755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ffc15d320 0x7f2ffc0d0b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009870 tx=0x7f2ff402a210 comp rx=0 tx=0).stop 2024-04-26T20:23:54.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.801+0000 7f300187b640 1 -- 172.21.15.37:0/2195615755 shutdown_connections 2024-04-26T20:23:54.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.801+0000 7f300187b640 1 --2- 172.21.15.37:0/2195615755 >> v2:172.21.15.134:3300/0 conn(0x7f2ffc0d1090 0x7f2ffc0d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.801+0000 7f300187b640 1 --2- 172.21.15.37:0/2195615755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ffc15d320 0x7f2ffc0d0b50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.801+0000 7f300187b640 1 -- 172.21.15.37:0/2195615755 >> 172.21.15.37:0/2195615755 conn(0x7f2ffc0d2f80 msgr2=0x7f2ffc0d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:54.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.801+0000 7f300187b640 1 -- 172.21.15.37:0/2195615755 shutdown_connections 2024-04-26T20:23:54.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.801+0000 7f300187b640 1 -- 172.21.15.37:0/2195615755 wait complete. 2024-04-26T20:23:54.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.802+0000 7f300187b640 1 Processor -- start 2024-04-26T20:23:54.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.803+0000 7f300187b640 1 -- start start 2024-04-26T20:23:54.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.803+0000 7f300187b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ffc0cf990 0x7f2ffc1f8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.803+0000 7f300187b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ffc0d1090 0x7f2ffc0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.803+0000 7f300187b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ffc15d320 0x7f2ffc15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.803+0000 7f300187b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc0ce550 con 0x7f2ffc0d1090 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.803+0000 7f300187b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc0b2cd0 con 0x7f2ffc15d320 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.803+0000 7f300187b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2ffc0ce6d0 con 0x7f2ffc0cf990 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.803+0000 7f2ffa7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ffc0d1090 0x7f2ffc0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.803+0000 7f2ffa7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ffc0d1090 0x7f2ffc0c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37262/0 (socket says 172.21.15.37:37262) 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.803+0000 7f2ffa7fc640 1 -- 172.21.15.37:0/3616424486 learned_addr learned my addr 172.21.15.37:0/3616424486 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f2ffb7fe640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ffc15d320 0x7f2ffc15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f2ffaffd640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ffc0cf990 0x7f2ffc1f8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f2ffa7fc640 1 -- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ffc0cf990 msgr2=0x7f2ffc1f8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f2ffa7fc640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ffc0cf990 0x7f2ffc1f8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f2ffa7fc640 1 -- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ffc15d320 msgr2=0x7f2ffc15acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f2ffa7fc640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ffc15d320 0x7f2ffc15acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f2ffa7fc640 1 -- 172.21.15.37:0/3616424486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc06da70 con 0x7f2ffc0d1090 2024-04-26T20:23:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f2ffa7fc640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ffc0d1090 0x7f2ffc0c9100 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000d800 tx=0x7f2ff000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f2fe3fff640 1 -- 172.21.15.37:0/3616424486 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff000b6c0 con 0x7f2ffc0d1090 2024-04-26T20:23:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f2fe3fff640 1 -- 172.21.15.37:0/3616424486 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff0019070 con 0x7f2ffc0d1090 2024-04-26T20:23:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f300187b640 1 -- 172.21.15.37:0/3616424486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc06dd50 con 0x7f2ffc0d1090 2024-04-26T20:23:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f2fe3fff640 1 -- 172.21.15.37:0/3616424486 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff000ba80 con 0x7f2ffc0d1090 2024-04-26T20:23:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.804+0000 7f300187b640 1 -- 172.21.15.37:0/3616424486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc06e310 con 0x7f2ffc0d1090 2024-04-26T20:23:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.805+0000 7f300187b640 1 -- 172.21.15.37:0/3616424486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ffc0693e0 con 0x7f2ffc0d1090 2024-04-26T20:23:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.806+0000 7f2fe3fff640 1 -- 172.21.15.37:0/3616424486 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f2ff000bc20 con 0x7f2ffc0d1090 2024-04-26T20:23:54.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.806+0000 7f2fe3fff640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2fe806cbc0 0x7f2fe806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.806+0000 7f2fe3fff640 1 -- 172.21.15.37:0/3616424486 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f2ff008c4b0 con 0x7f2ffc0d1090 2024-04-26T20:23:54.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.806+0000 7f2ffaffd640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2fe806cbc0 0x7f2fe806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.807+0000 7f2ffaffd640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2fe806cbc0 0x7f2fe806f080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4034000 tx=0x7f2ff402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:54.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.811+0000 7f2fe3fff640 1 -- 172.21.15.37:0/3616424486 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2ff005a570 con 0x7f2ffc0d1090 2024-04-26T20:23:54.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.996+0000 7f300187b640 1 -- 172.21.15.37:0/3616424486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2ffc0cd670 con 0x7f2ffc0d1090 2024-04-26T20:23:54.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.997+0000 7f2fe3fff640 1 -- 172.21.15.37:0/3616424486 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f2ff0059e70 con 0x7f2ffc0d1090 2024-04-26T20:23:54.998 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:23:55.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.998+0000 7f300187b640 1 -- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2fe806cbc0 msgr2=0x7f2fe806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.998+0000 7f300187b640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2fe806cbc0 0x7f2fe806f080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4034000 tx=0x7f2ff402e000 comp rx=0 tx=0).stop 2024-04-26T20:23:55.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.999+0000 7f300187b640 1 -- 172.21.15.37:0/3616424486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ffc0d1090 msgr2=0x7f2ffc0c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.999+0000 7f300187b640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ffc0d1090 0x7f2ffc0c9100 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000d800 tx=0x7f2ff000dcc0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.999+0000 7f300187b640 1 -- 172.21.15.37:0/3616424486 shutdown_connections 2024-04-26T20:23:55.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.999+0000 7f300187b640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ffc15d320 0x7f2ffc15acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.999+0000 7f300187b640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ffc0d1090 0x7f2ffc0c9100 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.999+0000 7f300187b640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2fe806cbc0 0x7f2fe806f080 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.999+0000 7f300187b640 1 --2- 172.21.15.37:0/3616424486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ffc0cf990 0x7f2ffc1f8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.999+0000 7f300187b640 1 -- 172.21.15.37:0/3616424486 >> 172.21.15.37:0/3616424486 conn(0x7f2ffc0d2f80 msgr2=0x7f2ffc15be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:55.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.999+0000 7f300187b640 1 -- 172.21.15.37:0/3616424486 shutdown_connections 2024-04-26T20:23:55.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:54.999+0000 7f300187b640 1 -- 172.21.15.37:0/3616424486 wait complete. 2024-04-26T20:23:55.010 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:23:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.114+0000 7fc1ce178640 1 Processor -- start 2024-04-26T20:23:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.114+0000 7fc1ce178640 1 -- start start 2024-04-26T20:23:55.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.114+0000 7fc1ce178640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c8160800 0x7fc1c81610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.114+0000 7fc1ce178640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc1c80d5f80 0x7fc1c80d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.114+0000 7fc1ce178640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c80d0930 con 0x7fc1c80d5b60 2024-04-26T20:23:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.114+0000 7fc1ce178640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c80d0c00 con 0x7fc1c80d5f80 2024-04-26T20:23:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.114+0000 7fc1ce178640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c80d0ee0 con 0x7fc1c8160800 2024-04-26T20:23:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.115+0000 7fc1c77fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c8160800 0x7fc1c81610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.115+0000 7fc1c7fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc1c80d5f80 0x7fc1c80d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.115+0000 7fc1c77fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c8160800 0x7fc1c81610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:37476/0 (socket says 172.21.15.37:37476) 2024-04-26T20:23:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.115+0000 7fc1c77fe640 1 -- 172.21.15.37:0/686933138 learned_addr learned my addr 172.21.15.37:0/686933138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.115+0000 7fc1c77fe640 1 -- 172.21.15.37:0/686933138 >> v2:172.21.15.134:3300/0 conn(0x7fc1c80d5f80 msgr2=0x7fc1c80d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.115+0000 7fc1c77fe640 1 --2- 172.21.15.37:0/686933138 >> v2:172.21.15.134:3300/0 conn(0x7fc1c80d5f80 0x7fc1c80d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.115+0000 7fc1c77fe640 1 -- 172.21.15.37:0/686933138 >> v1:172.21.15.134:6789/0 conn(0x7fc1c80d5b60 legacy=0x7fc1c80d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.115+0000 7fc1c77fe640 1 -- 172.21.15.37:0/686933138 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c80d11c0 con 0x7fc1c8160800 2024-04-26T20:23:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.115+0000 7fc1c7fff640 1 --2- 172.21.15.37:0/686933138 >> v2:172.21.15.134:3300/0 conn(0x7fc1c80d5f80 0x7fc1c80d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.116+0000 7fc1c77fe640 1 --2- 172.21.15.37:0/686933138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c8160800 0x7fc1c81610c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009870 tx=0x7fc1b802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2d7f56895052487 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.116+0000 7fc1c67fc640 1 -- 172.21.15.37:0/686933138 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8037070 con 0x7fc1c8160800 2024-04-26T20:23:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.116+0000 7fc1c67fc640 1 -- 172.21.15.37:0/686933138 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b8032440 con 0x7fc1c8160800 2024-04-26T20:23:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.116+0000 7fc1c67fc640 1 -- 172.21.15.37:0/686933138 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8032810 con 0x7fc1c8160800 2024-04-26T20:23:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.117+0000 7fc1ce178640 1 -- 172.21.15.37:0/686933138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c8160800 msgr2=0x7fc1c81610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.117+0000 7fc1ce178640 1 --2- 172.21.15.37:0/686933138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c8160800 0x7fc1c81610c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009870 tx=0x7fc1b802a270 comp rx=0 tx=0).stop 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.117+0000 7fc1ce178640 1 -- 172.21.15.37:0/686933138 shutdown_connections 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.117+0000 7fc1ce178640 1 --2- 172.21.15.37:0/686933138 >> v2:172.21.15.134:3300/0 conn(0x7fc1c80d5f80 0x7fc1c80d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.117+0000 7fc1ce178640 1 --2- 172.21.15.37:0/686933138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c8160800 0x7fc1c81610c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.117+0000 7fc1ce178640 1 -- 172.21.15.37:0/686933138 >> 172.21.15.37:0/686933138 conn(0x7fc1c809e550 msgr2=0x7fc1c80d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.117+0000 7fc1ce178640 1 -- 172.21.15.37:0/686933138 shutdown_connections 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.117+0000 7fc1ce178640 1 -- 172.21.15.37:0/686933138 wait complete. 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.118+0000 7fc1ce178640 1 Processor -- start 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.118+0000 7fc1ce178640 1 -- start start 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1ce178640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c80d5b60 0x7fc1c80ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1ce178640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c8160800 0x7fc1c80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1ce178640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c80cd720 0x7fc1c8156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1ce178640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c80b6300 con 0x7fc1c8160800 2024-04-26T20:23:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1ce178640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c80d30f0 con 0x7fc1c80cd720 2024-04-26T20:23:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1ce178640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c8160be0 con 0x7fc1c80d5b60 2024-04-26T20:23:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c8160800 0x7fc1c80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c8160800 0x7fc1c80ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37274/0 (socket says 172.21.15.37:37274) 2024-04-26T20:23:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c6ffd640 1 -- 172.21.15.37:0/616131565 learned_addr learned my addr 172.21.15.37:0/616131565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c7fff640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c80cd720 0x7fc1c8156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c77fe640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c80d5b60 0x7fc1c80ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c6ffd640 1 -- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c80d5b60 msgr2=0x7fc1c80ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c6ffd640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c80d5b60 0x7fc1c80ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c6ffd640 1 -- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c80cd720 msgr2=0x7fc1c8156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c6ffd640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c80cd720 0x7fc1c8156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c6ffd640 1 -- 172.21.15.37:0/616131565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c80cc680 con 0x7fc1c8160800 2024-04-26T20:23:55.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c7fff640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c80cd720 0x7fc1c8156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:55.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c77fe640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c80d5b60 0x7fc1c80ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:55.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.119+0000 7fc1c6ffd640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c8160800 0x7fc1c80ceac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00c400 tx=0x7fc1bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:55.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.120+0000 7fc1affff640 1 -- 172.21.15.37:0/616131565 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0052d0 con 0x7fc1c8160800 2024-04-26T20:23:55.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.120+0000 7fc1affff640 1 -- 172.21.15.37:0/616131565 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc005490 con 0x7fc1c8160800 2024-04-26T20:23:55.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.120+0000 7fc1ce178640 1 -- 172.21.15.37:0/616131565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c80cc960 con 0x7fc1c8160800 2024-04-26T20:23:55.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.120+0000 7fc1affff640 1 -- 172.21.15.37:0/616131565 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc007260 con 0x7fc1c8160800 2024-04-26T20:23:55.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.120+0000 7fc1ce178640 1 -- 172.21.15.37:0/616131565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c80ccf20 con 0x7fc1c8160800 2024-04-26T20:23:55.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.121+0000 7fc1ce178640 1 -- 172.21.15.37:0/616131565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c806a360 con 0x7fc1c8160800 2024-04-26T20:23:55.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.121+0000 7fc1affff640 1 -- 172.21.15.37:0/616131565 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fc1bc007400 con 0x7fc1c8160800 2024-04-26T20:23:55.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.122+0000 7fc1affff640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1b406cbc0 0x7fc1b406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.122+0000 7fc1affff640 1 -- 172.21.15.37:0/616131565 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7fc1bc08b650 con 0x7fc1c8160800 2024-04-26T20:23:55.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.122+0000 7fc1c77fe640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1b406cbc0 0x7fc1b406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.122+0000 7fc1c77fe640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1b406cbc0 0x7fc1b406f080 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8034000 tx=0x7fc1b802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:55.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.126+0000 7fc1affff640 1 -- 172.21.15.37:0/616131565 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1bc059630 con 0x7fc1c8160800 2024-04-26T20:23:55.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.311+0000 7fc1ce178640 1 -- 172.21.15.37:0/616131565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc1c80d2960 con 0x7fc1c8160800 2024-04-26T20:23:55.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.311+0000 7fc1affff640 1 -- 172.21.15.37:0/616131565 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7fc1bc058f30 con 0x7fc1c8160800 2024-04-26T20:23:55.312 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:23:55.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 -- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1b406cbc0 msgr2=0x7fc1b406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1b406cbc0 0x7fc1b406f080 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8034000 tx=0x7fc1b802e000 comp rx=0 tx=0).stop 2024-04-26T20:23:55.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 -- 172.21.15.37:0/616131565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c8160800 msgr2=0x7fc1c80ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c8160800 0x7fc1c80ceac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00c400 tx=0x7fc1bc00c8c0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 -- 172.21.15.37:0/616131565 shutdown_connections 2024-04-26T20:23:55.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c80cd720 0x7fc1c8156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1c8160800 0x7fc1c80ceac0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc1b406cbc0 0x7fc1b406f080 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 --2- 172.21.15.37:0/616131565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c80d5b60 0x7fc1c80ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 -- 172.21.15.37:0/616131565 >> 172.21.15.37:0/616131565 conn(0x7fc1c809e550 msgr2=0x7fc1c815ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:55.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 -- 172.21.15.37:0/616131565 shutdown_connections 2024-04-26T20:23:55.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.313+0000 7fc1ce178640 1 -- 172.21.15.37:0/616131565 wait complete. 2024-04-26T20:23:55.325 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:23:55.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.425+0000 7fa9dd209640 1 Processor -- start 2024-04-26T20:23:55.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.425+0000 7fa9dd209640 1 -- start start 2024-04-26T20:23:55.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.425+0000 7fa9dd209640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d8162b40 0x7fa9d80d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.425+0000 7fa9dd209640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa9d80d3820 0x7fa9d80d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.425+0000 7fa9dd209640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d80d0210 con 0x7fa9d80d1850 2024-04-26T20:23:55.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.425+0000 7fa9dd209640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa9d80d41c0 con 0x7fa9d80d3820 2024-04-26T20:23:55.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.425+0000 7fa9dd209640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d80d04f0 con 0x7fa9d8162b40 2024-04-26T20:23:55.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.426+0000 7fa9d6d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d8162b40 0x7fa9d80d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.426+0000 7fa9cffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa9d80d3820 0x7fa9d80d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.426+0000 7fa9d6d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d8162b40 0x7fa9d80d5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:37484/0 (socket says 172.21.15.37:37484) 2024-04-26T20:23:55.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.426+0000 7fa9d6d9d640 1 -- 172.21.15.37:0/3568525177 learned_addr learned my addr 172.21.15.37:0/3568525177 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:55.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.426+0000 7fa9d6d9d640 1 -- 172.21.15.37:0/3568525177 >> v2:172.21.15.134:3300/0 conn(0x7fa9d80d3820 msgr2=0x7fa9d80d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:55.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.426+0000 7fa9d6d9d640 1 --2- 172.21.15.37:0/3568525177 >> v2:172.21.15.134:3300/0 conn(0x7fa9d80d3820 0x7fa9d80d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.426+0000 7fa9d6d9d640 1 -- 172.21.15.37:0/3568525177 >> v1:172.21.15.134:6789/0 conn(0x7fa9d80d1850 legacy=0x7fa9d80d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.426+0000 7fa9d6d9d640 1 -- 172.21.15.37:0/3568525177 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d80d07d0 con 0x7fa9d8162b40 2024-04-26T20:23:55.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.427+0000 7fa9d6d9d640 1 --2- 172.21.15.37:0/3568525177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d8162b40 0x7fa9d80d5bd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009870 tx=0x7fa9c802a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e12c5ef8cccd19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:55.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.427+0000 7fa9d659c640 1 -- 172.21.15.37:0/3568525177 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8037070 con 0x7fa9d8162b40 2024-04-26T20:23:55.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.427+0000 7fa9d659c640 1 -- 172.21.15.37:0/3568525177 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c802ae50 con 0x7fa9d8162b40 2024-04-26T20:23:55.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.427+0000 7fa9d659c640 1 -- 172.21.15.37:0/3568525177 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8032860 con 0x7fa9d8162b40 2024-04-26T20:23:55.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.427+0000 7fa9dd209640 1 -- 172.21.15.37:0/3568525177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d8162b40 msgr2=0x7fa9d80d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.427+0000 7fa9dd209640 1 --2- 172.21.15.37:0/3568525177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d8162b40 0x7fa9d80d5bd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009870 tx=0x7fa9c802a300 comp rx=0 tx=0).stop 2024-04-26T20:23:55.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.428+0000 7fa9dd209640 1 -- 172.21.15.37:0/3568525177 shutdown_connections 2024-04-26T20:23:55.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.428+0000 7fa9dd209640 1 --2- 172.21.15.37:0/3568525177 >> v2:172.21.15.134:3300/0 conn(0x7fa9d80d3820 0x7fa9d80d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.428+0000 7fa9dd209640 1 --2- 172.21.15.37:0/3568525177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d8162b40 0x7fa9d80d5bd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.428+0000 7fa9dd209640 1 -- 172.21.15.37:0/3568525177 >> 172.21.15.37:0/3568525177 conn(0x7fa9d809e550 msgr2=0x7fa9d80d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:55.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.428+0000 7fa9dd209640 1 -- 172.21.15.37:0/3568525177 shutdown_connections 2024-04-26T20:23:55.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.428+0000 7fa9dd209640 1 -- 172.21.15.37:0/3568525177 wait complete. 2024-04-26T20:23:55.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.429+0000 7fa9dd209640 1 Processor -- start 2024-04-26T20:23:55.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.429+0000 7fa9dd209640 1 -- start start 2024-04-26T20:23:55.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9dd209640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d80d1850 0x7fa9d80cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9dd209640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9d80d3820 0x7fa9d80b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9dd209640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8162b40 0x7fa9d80ba840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9dd209640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d80b5bb0 con 0x7fa9d80d3820 2024-04-26T20:23:55.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9dd209640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d80d0d90 con 0x7fa9d8162b40 2024-04-26T20:23:55.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9dd209640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d80d6110 con 0x7fa9d80d1850 2024-04-26T20:23:55.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9d80d3820 0x7fa9d80b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9d80d3820 0x7fa9d80b8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37280/0 (socket says 172.21.15.37:37280) 2024-04-26T20:23:55.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cffff640 1 -- 172.21.15.37:0/2374583675 learned_addr learned my addr 172.21.15.37:0/2374583675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:55.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9d6d9d640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d80d1850 0x7fa9d80cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9d759e640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8162b40 0x7fa9d80ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cffff640 1 -- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d80d1850 msgr2=0x7fa9d80cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cffff640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d80d1850 0x7fa9d80cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cffff640 1 -- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8162b40 msgr2=0x7fa9d80ba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cffff640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8162b40 0x7fa9d80ba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cffff640 1 -- 172.21.15.37:0/2374583675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d80b9520 con 0x7fa9d80d3820 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cffff640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9d80d3820 0x7fa9d80b8fe0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000e870 tx=0x7fa9c000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cf7fe640 1 -- 172.21.15.37:0/2374583675 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0017070 con 0x7fa9d80d3820 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cf7fe640 1 -- 172.21.15.37:0/2374583675 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c000c940 con 0x7fa9d80d3820 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9dd209640 1 -- 172.21.15.37:0/2374583675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d80b97a0 con 0x7fa9d80d3820 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.430+0000 7fa9cf7fe640 1 -- 172.21.15.37:0/2374583675 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000cd30 con 0x7fa9d80d3820 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.431+0000 7fa9dd209640 1 -- 172.21.15.37:0/2374583675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d80ba040 con 0x7fa9d80d3820 2024-04-26T20:23:55.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.431+0000 7fa9dd209640 1 -- 172.21.15.37:0/2374583675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d806a360 con 0x7fa9d80d3820 2024-04-26T20:23:55.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.432+0000 7fa9cf7fe640 1 -- 172.21.15.37:0/2374583675 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fa9c001a070 con 0x7fa9d80d3820 2024-04-26T20:23:55.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.432+0000 7fa9cf7fe640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9b806cbc0 0x7fa9b806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.433+0000 7fa9cf7fe640 1 -- 172.21.15.37:0/2374583675 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7fa9c008be20 con 0x7fa9d80d3820 2024-04-26T20:23:55.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.433+0000 7fa9d6d9d640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9b806cbc0 0x7fa9b806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.433+0000 7fa9d6d9d640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9b806cbc0 0x7fa9b806f080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8034000 tx=0x7fa9c802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:55.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.437+0000 7fa9cf7fe640 1 -- 172.21.15.37:0/2374583675 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9d80b9520 con 0x7fa9d80d3820 2024-04-26T20:23:55.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.660+0000 7fa9dd209640 1 -- 172.21.15.37:0/2374583675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9d80d26f0 con 0x7fa9d80d3820 2024-04-26T20:23:55.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.660+0000 7fa9cf7fe640 1 -- 172.21.15.37:0/2374583675 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5181 (secure 0 0 0) 0x7fa9c0059890 con 0x7fa9d80d3820 2024-04-26T20:23:55.661 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:23:55.662 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/4166977617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4166977617},{"type":"v1","addr":"172.21.15.134:6836","nonce":4166977617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3245561282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3245561282},{"type":"v1","addr":"172.21.15.134:6837","nonce":3245561282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3853963851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3853963851},{"type":"v1","addr":"172.21.15.37:6835","nonce":3853963851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:22:42.116910+0000","modified":"2024-04-26T20:22:45.119356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5097":{"gid":5097,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/64661056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":64661056},{"type":"v1","addr":"172.21.15.37:6834","nonce":64661056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5097,"qdb_cluster":[5097]},"id":2}]} 2024-04-26T20:23:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 16 2024-04-26T20:23:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 -- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9b806cbc0 msgr2=0x7fa9b806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9b806cbc0 0x7fa9b806f080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8034000 tx=0x7fa9c802e000 comp rx=0 tx=0).stop 2024-04-26T20:23:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 -- 172.21.15.37:0/2374583675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9d80d3820 msgr2=0x7fa9d80b8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9d80d3820 0x7fa9d80b8fe0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000e870 tx=0x7fa9c000ed30 comp rx=0 tx=0).stop 2024-04-26T20:23:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 -- 172.21.15.37:0/2374583675 shutdown_connections 2024-04-26T20:23:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8162b40 0x7fa9d80ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9d80d3820 0x7fa9d80b8fe0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa9b806cbc0 0x7fa9b806f080 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 --2- 172.21.15.37:0/2374583675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9d80d1850 0x7fa9d80cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 -- 172.21.15.37:0/2374583675 >> 172.21.15.37:0/2374583675 conn(0x7fa9d809e550 msgr2=0x7fa9d80b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 -- 172.21.15.37:0/2374583675 shutdown_connections 2024-04-26T20:23:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.662+0000 7fa9dd209640 1 -- 172.21.15.37:0/2374583675 wait complete. 2024-04-26T20:23:55.674 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:23:55.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.774+0000 7f04777bf640 1 Processor -- start 2024-04-26T20:23:55.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.774+0000 7f04777bf640 1 -- start start 2024-04-26T20:23:55.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f04777bf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f047006f790 0x7f0470074030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f04777bf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f04700b6b10 0x7f04701611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f04777bf640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0470161b80 con 0x7f0470160d70 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f04777bf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f0470161710 con 0x7f04700b6b10 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f04777bf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f04700b4400 con 0x7f047006f790 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f0475d5c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f04700b6b10 0x7f04701611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f0475d5c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f04700b6b10 0x7f04701611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59234/0 (socket says 172.21.15.37:59234) 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f0475d5c640 1 -- 172.21.15.37:0/2645958009 learned_addr learned my addr 172.21.15.37:0/2645958009 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f047555b640 1 --2- 172.21.15.37:0/2645958009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f047006f790 0x7f0470074030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f0475d5c640 1 -- 172.21.15.37:0/2645958009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f047006f790 msgr2=0x7f0470074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f0475d5c640 1 --2- 172.21.15.37:0/2645958009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f047006f790 0x7f0470074030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f0475d5c640 1 -- 172.21.15.37:0/2645958009 >> v1:172.21.15.134:6789/0 conn(0x7f0470160d70 legacy=0x7f047015d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.775+0000 7f0475d5c640 1 -- 172.21.15.37:0/2645958009 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0470074570 con 0x7f04700b6b10 2024-04-26T20:23:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.776+0000 7f0475d5c640 1 --2- 172.21.15.37:0/2645958009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04700b6b10 0x7f04701611d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0468009e50 tx=0x7f046802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d2fc26d2085274 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.776+0000 7f0467fff640 1 -- 172.21.15.37:0/2645958009 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468036070 con 0x7f04700b6b10 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.776+0000 7f0467fff640 1 -- 172.21.15.37:0/2645958009 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0468031600 con 0x7f04700b6b10 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.776+0000 7f0467fff640 1 -- 172.21.15.37:0/2645958009 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680319d0 con 0x7f04700b6b10 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.776+0000 7f04777bf640 1 -- 172.21.15.37:0/2645958009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04700b6b10 msgr2=0x7f04701611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.776+0000 7f04777bf640 1 --2- 172.21.15.37:0/2645958009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04700b6b10 0x7f04701611d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0468009e50 tx=0x7f046802a710 comp rx=0 tx=0).stop 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.777+0000 7f04777bf640 1 -- 172.21.15.37:0/2645958009 shutdown_connections 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.777+0000 7f04777bf640 1 --2- 172.21.15.37:0/2645958009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04700b6b10 0x7f04701611d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.777+0000 7f04777bf640 1 --2- 172.21.15.37:0/2645958009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f047006f790 0x7f0470074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.777+0000 7f04777bf640 1 -- 172.21.15.37:0/2645958009 >> 172.21.15.37:0/2645958009 conn(0x7f04700d0d50 msgr2=0x7f04700d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.777+0000 7f04777bf640 1 -- 172.21.15.37:0/2645958009 shutdown_connections 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.777+0000 7f04777bf640 1 -- 172.21.15.37:0/2645958009 wait complete. 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.778+0000 7f04777bf640 1 Processor -- start 2024-04-26T20:23:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.778+0000 7f04777bf640 1 -- start start 2024-04-26T20:23:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f04777bf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f047006f790 0x7f0470156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f04777bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04700b6b10 0x7f04700d0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f04777bf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0470160d70 0x7f04700ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f04777bf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04700732d0 con 0x7f04700b6b10 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f04777bf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0470072ff0 con 0x7f047006f790 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f04777bf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f04700735b0 con 0x7f0470160d70 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0474d5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04700b6b10 0x7f04700d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0474d5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04700b6b10 0x7f04700d0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59244/0 (socket says 172.21.15.37:59244) 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0474d5a640 1 -- 172.21.15.37:0/1498941678 learned_addr learned my addr 172.21.15.37:0/1498941678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f047555b640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f047006f790 0x7f0470156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0475d5c640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0470160d70 0x7f04700ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0474d5a640 1 -- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0470160d70 msgr2=0x7f04700ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0474d5a640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0470160d70 0x7f04700ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0474d5a640 1 -- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f047006f790 msgr2=0x7f0470156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0474d5a640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f047006f790 0x7f0470156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0474d5a640 1 -- 172.21.15.37:0/1498941678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04700cf490 con 0x7f04700b6b10 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f047555b640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f047006f790 0x7f0470156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0474d5a640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04700b6b10 0x7f04700d0530 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f046000c430 tx=0x7f046000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:55.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0465ffb640 1 -- 172.21.15.37:0/1498941678 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04600052d0 con 0x7f04700b6b10 2024-04-26T20:23:55.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0465ffb640 1 -- 172.21.15.37:0/1498941678 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0460005490 con 0x7f04700b6b10 2024-04-26T20:23:55.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f04777bf640 1 -- 172.21.15.37:0/1498941678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04700cf770 con 0x7f04700b6b10 2024-04-26T20:23:55.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.779+0000 7f0465ffb640 1 -- 172.21.15.37:0/1498941678 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460007260 con 0x7f04700b6b10 2024-04-26T20:23:55.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.780+0000 7f04777bf640 1 -- 172.21.15.37:0/1498941678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04700cfd30 con 0x7f04700b6b10 2024-04-26T20:23:55.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.781+0000 7f04777bf640 1 -- 172.21.15.37:0/1498941678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f047006a360 con 0x7f04700b6b10 2024-04-26T20:23:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.781+0000 7f0465ffb640 1 -- 172.21.15.37:0/1498941678 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f0460007400 con 0x7f04700b6b10 2024-04-26T20:23:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.781+0000 7f0465ffb640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f045406cbc0 0x7f045406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.781+0000 7f0465ffb640 1 -- 172.21.15.37:0/1498941678 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f046008b3e0 con 0x7f04700b6b10 2024-04-26T20:23:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.782+0000 7f047555b640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f045406cbc0 0x7f045406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.782+0000 7f047555b640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f045406cbc0 0x7f045406f080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f046c009af0 tx=0x7f046c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:55.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.786+0000 7f0465ffb640 1 -- 172.21.15.37:0/1498941678 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0460058ee0 con 0x7f04700b6b10 2024-04-26T20:23:55.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.971+0000 7f04777bf640 1 -- 172.21.15.37:0/1498941678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f047006ff40 con 0x7f04700b6b10 2024-04-26T20:23:55.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.973+0000 7f0465ffb640 1 -- 172.21.15.37:0/1498941678 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16746 (secure 0 0 0) 0x7f046005d1e0 con 0x7f04700b6b10 2024-04-26T20:23:55.974 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:23:55.974 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:23:34.988080+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:22:39.762703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:22:41.066360+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:23:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 -- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f045406cbc0 msgr2=0x7f045406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f045406cbc0 0x7f045406f080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f046c009af0 tx=0x7f046c008040 comp rx=0 tx=0).stop 2024-04-26T20:23:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 -- 172.21.15.37:0/1498941678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04700b6b10 msgr2=0x7f04700d0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04700b6b10 0x7f04700d0530 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f046000c430 tx=0x7f046000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 -- 172.21.15.37:0/1498941678 shutdown_connections 2024-04-26T20:23:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f045406cbc0 0x7f045406f080 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0470160d70 0x7f04700ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04700b6b10 0x7f04700d0530 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 --2- 172.21.15.37:0/1498941678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f047006f790 0x7f0470156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 -- 172.21.15.37:0/1498941678 >> 172.21.15.37:0/1498941678 conn(0x7f04700d0d50 msgr2=0x7f0470072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 -- 172.21.15.37:0/1498941678 shutdown_connections 2024-04-26T20:23:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:55.975+0000 7f04777bf640 1 -- 172.21.15.37:0/1498941678 wait complete. 2024-04-26T20:23:55.987 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:23:55.987 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:23:56.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.087+0000 7f23fc247640 1 Processor -- start 2024-04-26T20:23:56.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.087+0000 7f23fc247640 1 -- start start 2024-04-26T20:23:56.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23fc247640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f23f406ea40 0x7f23f40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23fc247640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23f40cf830 0x7f23f40cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23fc247640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23f40b1d70 con 0x7f23f4162e40 2024-04-26T20:23:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23fc247640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f23f4163220 con 0x7f23f406ea40 2024-04-26T20:23:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23fc247640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f23f40d01c0 con 0x7f23f40cf830 2024-04-26T20:23:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23f97e2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f23f406ea40 0x7f23f40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23f97e2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f23f406ea40 0x7f23f40d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59248/0 (socket says 172.21.15.37:59248) 2024-04-26T20:23:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23f97e2640 1 -- 172.21.15.37:0/3057512271 learned_addr learned my addr 172.21.15.37:0/3057512271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23fa7e4640 1 --2- 172.21.15.37:0/3057512271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23f40cf830 0x7f23f40cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23f97e2640 1 -- 172.21.15.37:0/3057512271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23f40cf830 msgr2=0x7f23f40cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23f97e2640 1 --2- 172.21.15.37:0/3057512271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23f40cf830 0x7f23f40cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23f97e2640 1 -- 172.21.15.37:0/3057512271 >> v1:172.21.15.134:6789/0 conn(0x7f23f4162e40 legacy=0x7f23f40b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.088+0000 7f23f97e2640 1 -- 172.21.15.37:0/3057512271 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f40d1960 con 0x7f23f406ea40 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.089+0000 7f23f97e2640 1 --2- 172.21.15.37:0/3057512271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f406ea40 0x7f23f40d1340 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009870 tx=0x7f23e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f82a72f778dc45 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.089+0000 7f23f8fe1640 1 -- 172.21.15.37:0/3057512271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4037070 con 0x7f23f406ea40 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.089+0000 7f23f8fe1640 1 -- 172.21.15.37:0/3057512271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e4032440 con 0x7f23f406ea40 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.089+0000 7f23f8fe1640 1 -- 172.21.15.37:0/3057512271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4032810 con 0x7f23f406ea40 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.090+0000 7f23fc247640 1 -- 172.21.15.37:0/3057512271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f406ea40 msgr2=0x7f23f40d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.090+0000 7f23fc247640 1 --2- 172.21.15.37:0/3057512271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f406ea40 0x7f23f40d1340 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009870 tx=0x7f23e402a180 comp rx=0 tx=0).stop 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.090+0000 7f23fc247640 1 -- 172.21.15.37:0/3057512271 shutdown_connections 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.090+0000 7f23fc247640 1 --2- 172.21.15.37:0/3057512271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23f40cf830 0x7f23f40cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.090+0000 7f23fc247640 1 --2- 172.21.15.37:0/3057512271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f406ea40 0x7f23f40d1340 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.090+0000 7f23fc247640 1 -- 172.21.15.37:0/3057512271 >> 172.21.15.37:0/3057512271 conn(0x7f23f409ba50 msgr2=0x7f23f409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.090+0000 7f23fc247640 1 -- 172.21.15.37:0/3057512271 shutdown_connections 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.090+0000 7f23fc247640 1 -- 172.21.15.37:0/3057512271 wait complete. 2024-04-26T20:23:56.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.091+0000 7f23fc247640 1 Processor -- start 2024-04-26T20:23:56.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.091+0000 7f23fc247640 1 -- start start 2024-04-26T20:23:56.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fc247640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23f406ea40 0x7f23f40b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fc247640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23f40cf830 0x7f23f41f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fc247640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f4162e40 0x7f23f415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fc247640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23f40d1f20 con 0x7f23f4162e40 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fc247640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f23f40d0eb0 con 0x7f23f406ea40 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fc247640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f23f40ce010 con 0x7f23f40cf830 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fa7e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f4162e40 0x7f23f415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fa7e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f4162e40 0x7f23f415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59264/0 (socket says 172.21.15.37:59264) 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fa7e4640 1 -- 172.21.15.37:0/141193272 learned_addr learned my addr 172.21.15.37:0/141193272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fa7e4640 1 -- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23f40cf830 msgr2=0x7f23f41f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fa7e4640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23f40cf830 0x7f23f41f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fa7e4640 1 -- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23f406ea40 msgr2=0x7f23f40b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fa7e4640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23f406ea40 0x7f23f40b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fa7e4640 1 -- 172.21.15.37:0/141193272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f41f6210 con 0x7f23f4162e40 2024-04-26T20:23:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fa7e4640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f4162e40 0x7f23f415ad90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f23ec007e40 tx=0x7f23ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:56.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23ea7fc640 1 -- 172.21.15.37:0/141193272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec014620 con 0x7f23f4162e40 2024-04-26T20:23:56.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23ea7fc640 1 -- 172.21.15.37:0/141193272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23ec019070 con 0x7f23f4162e40 2024-04-26T20:23:56.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fc247640 1 -- 172.21.15.37:0/141193272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f40b01c0 con 0x7f23f4162e40 2024-04-26T20:23:56.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23ea7fc640 1 -- 172.21.15.37:0/141193272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec014a20 con 0x7f23f4162e40 2024-04-26T20:23:56.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.092+0000 7f23fc247640 1 -- 172.21.15.37:0/141193272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f40b0780 con 0x7f23f4162e40 2024-04-26T20:23:56.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.093+0000 7f23fc247640 1 -- 172.21.15.37:0/141193272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f406a360 con 0x7f23f4162e40 2024-04-26T20:23:56.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.094+0000 7f23ea7fc640 1 -- 172.21.15.37:0/141193272 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f23ec014bc0 con 0x7f23f4162e40 2024-04-26T20:23:56.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.094+0000 7f23ea7fc640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f23d806cbc0 0x7f23d806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.094+0000 7f23ea7fc640 1 -- 172.21.15.37:0/141193272 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f23ec08c900 con 0x7f23f4162e40 2024-04-26T20:23:56.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.095+0000 7f23f9fe3640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f23d806cbc0 0x7f23d806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.095+0000 7f23f9fe3640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f23d806cbc0 0x7f23d806f080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23f0006fd0 tx=0x7f23f0006bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:56.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.099+0000 7f23ea7fc640 1 -- 172.21.15.37:0/141193272 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f23ec05a9c0 con 0x7f23f4162e40 2024-04-26T20:23:56.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.322+0000 7f23fc247640 1 -- 172.21.15.37:0/141193272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23f4161aa0 con 0x7f23f4162e40 2024-04-26T20:23:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.322+0000 7f23ea7fc640 1 -- 172.21.15.37:0/141193272 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5181 (secure 0 0 0) 0x7f23ec05a2c0 con 0x7f23f4162e40 2024-04-26T20:23:56.324 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:23:56.324 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/4166977617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4166977617},{"type":"v1","addr":"172.21.15.134:6836","nonce":4166977617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3245561282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3245561282},{"type":"v1","addr":"172.21.15.134:6837","nonce":3245561282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3853963851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3853963851},{"type":"v1","addr":"172.21.15.37:6835","nonce":3853963851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:22:42.116910+0000","modified":"2024-04-26T20:22:45.119356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5097":{"gid":5097,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/64661056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":64661056},{"type":"v1","addr":"172.21.15.37:6834","nonce":64661056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5097,"qdb_cluster":[5097]},"id":2}]} 2024-04-26T20:23:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 16 2024-04-26T20:23:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 -- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f23d806cbc0 msgr2=0x7f23d806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f23d806cbc0 0x7f23d806f080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23f0006fd0 tx=0x7f23f0006bc0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 -- 172.21.15.37:0/141193272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f4162e40 msgr2=0x7f23f415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f4162e40 0x7f23f415ad90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f23ec007e40 tx=0x7f23ec00e4b0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 -- 172.21.15.37:0/141193272 shutdown_connections 2024-04-26T20:23:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f4162e40 0x7f23f415ad90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f23d806cbc0 0x7f23d806f080 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23f40cf830 0x7f23f41f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 --2- 172.21.15.37:0/141193272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23f406ea40 0x7f23f40b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 -- 172.21.15.37:0/141193272 >> 172.21.15.37:0/141193272 conn(0x7f23f409ba50 msgr2=0x7f23f40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 -- 172.21.15.37:0/141193272 shutdown_connections 2024-04-26T20:23:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.324+0000 7f23fc247640 1 -- 172.21.15.37:0/141193272 wait complete. 2024-04-26T20:23:56.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:23:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.436+0000 7f7b5019d640 1 Processor -- start 2024-04-26T20:23:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.436+0000 7f7b5019d640 1 -- start start 2024-04-26T20:23:56.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b5019d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b480d1c40 0x7f7b4806e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b5019d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b480d2020 con 0x7f7b480cf810 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b5019d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b480ce700 con 0x7f7b4806fe90 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b5019d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4806ea70 con 0x7f7b480d1c40 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4e73a640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7b480cf810 0x7f7b480cbfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33170/0 (socket says 172.21.15.37:33170) 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4e73a640 1 -- 172.21.15.37:0/1862359074 learned_addr learned my addr 172.21.15.37:0/1862359074 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4df39640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b480d1c40 0x7f7b4806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744811921 0 0) 0x7f7b480d2020 con 0x7f7b480cf810 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b34003520 con 0x7f7b480cf810 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2997657154 0 0) 0x7f7b480ce700 con 0x7f7b4806fe90 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b480d2020 con 0x7f7b4806fe90 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2189145963 0 0) 0x7f7b34003520 con 0x7f7b480cf810 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b480d1c40 msgr2=0x7f7b4806e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4cf37640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b480d1c40 0x7f7b4806e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 >> v1:172.21.15.134:6789/0 conn(0x7f7b4806fe90 legacy=0x7f7b480cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b480ce9e0 con 0x7f7b480cf810 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f7b400019f0 con 0x7f7b480cf810 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.437+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 >> v1:172.21.15.37:6789/0 conn(0x7f7b480cf810 legacy=0x7f7b480cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4cf37640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b34003b70 0x7f7b34003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4cf37640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b34004800 0x7f7b34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4cf37640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b480d1c40 0x7f7b340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4806ea70 con 0x7f7b34003b70 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3400ba70 con 0x7f7b34004800 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7b3400bc40 con 0x7f7b480d1c40 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4df39640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b34003b70 0x7f7b34003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4d738640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b34004800 0x7f7b34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4e73a640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b480d1c40 0x7f7b340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4df39640 1 -- 172.21.15.37:0/1862359074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b480d1c40 msgr2=0x7f7b340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4df39640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b480d1c40 0x7f7b340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4df39640 1 -- 172.21.15.37:0/1862359074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b34004800 msgr2=0x7f7b34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4df39640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b34004800 0x7f7b34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.438+0000 7f7b4df39640 1 -- 172.21.15.37:0/1862359074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b34003520 con 0x7f7b34003b70 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.439+0000 7f7b4df39640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b34003b70 0x7f7b34003f70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009ac0 tx=0x7f7b34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc912edecd1ae23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.439+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44036070 con 0x7f7b34003b70 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.439+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b44031400 con 0x7f7b34003b70 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.439+0000 7f7b4cf37640 1 -- 172.21.15.37:0/1862359074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44009690 con 0x7f7b34003b70 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.439+0000 7f7b5019d640 1 -- 172.21.15.37:0/1862359074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b34003b70 msgr2=0x7f7b34003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.439+0000 7f7b5019d640 1 --2- 172.21.15.37:0/1862359074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b34003b70 0x7f7b34003f70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009ac0 tx=0x7f7b34005830 comp rx=0 tx=0).stop 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.439+0000 7f7b4e73a640 1 -- 172.21.15.37:0/1862359074 reap_dead start 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.439+0000 7f7b5019d640 1 -- 172.21.15.37:0/1862359074 shutdown_connections 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.439+0000 7f7b5019d640 1 -- 172.21.15.37:0/1862359074 >> 172.21.15.37:0/1862359074 conn(0x7f7b48073880 msgr2=0x7f7b48073c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.439+0000 7f7b5019d640 1 -- 172.21.15.37:0/1862359074 shutdown_connections 2024-04-26T20:23:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.439+0000 7f7b5019d640 1 -- 172.21.15.37:0/1862359074 wait complete. 2024-04-26T20:23:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.441+0000 7f7b5019d640 1 Processor -- start 2024-04-26T20:23:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.441+0000 7f7b5019d640 1 -- start start 2024-04-26T20:23:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.441+0000 7f7b5019d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34003b70 0x7f7b4809ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.441+0000 7f7b5019d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b480c9d50 0x7f7b480c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.441+0000 7f7b5019d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b480c8810 0x7f7b480af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.441+0000 7f7b5019d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b480cb850 con 0x7f7b480c9d50 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.441+0000 7f7b5019d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b480ce4b0 con 0x7f7b480c8810 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.441+0000 7f7b5019d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7b480cbb30 con 0x7f7b34003b70 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.441+0000 7f7b4d738640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b480c9d50 0x7f7b480c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b4d738640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b480c9d50 0x7f7b480c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59276/0 (socket says 172.21.15.37:59276) 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b4d738640 1 -- 172.21.15.37:0/3524886288 learned_addr learned my addr 172.21.15.37:0/3524886288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b4df39640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34003b70 0x7f7b4809ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b4e73a640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b480c8810 0x7f7b480af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b4d738640 1 -- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34003b70 msgr2=0x7f7b4809ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b4d738640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34003b70 0x7f7b4809ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b4d738640 1 -- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b480c8810 msgr2=0x7f7b480af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b4d738640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b480c8810 0x7f7b480af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b4d738640 1 -- 172.21.15.37:0/3524886288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4809f0f0 con 0x7f7b480c9d50 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b4e73a640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b480c8810 0x7f7b480af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b4d738640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b480c9d50 0x7f7b480c82d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800f7a0 tx=0x7f7b3800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b3e7fc640 1 -- 172.21.15.37:0/3524886288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800fe50 con 0x7f7b480c9d50 2024-04-26T20:23:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b3e7fc640 1 -- 172.21.15.37:0/3524886288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b38018070 con 0x7f7b480c9d50 2024-04-26T20:23:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b5019d640 1 -- 172.21.15.37:0/3524886288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b480c8cb0 con 0x7f7b480c9d50 2024-04-26T20:23:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b3e7fc640 1 -- 172.21.15.37:0/3524886288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800d730 con 0x7f7b480c9d50 2024-04-26T20:23:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.442+0000 7f7b5019d640 1 -- 172.21.15.37:0/3524886288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b480c9270 con 0x7f7b480c9d50 2024-04-26T20:23:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.443+0000 7f7b5019d640 1 -- 172.21.15.37:0/3524886288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b4806a360 con 0x7f7b480c9d50 2024-04-26T20:23:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.444+0000 7f7b3e7fc640 1 -- 172.21.15.37:0/3524886288 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f7b3800d8d0 con 0x7f7b480c9d50 2024-04-26T20:23:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.444+0000 7f7b3e7fc640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7b2c06cbc0 0x7f7b2c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.444+0000 7f7b3e7fc640 1 -- 172.21.15.37:0/3524886288 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f7b3808bfc0 con 0x7f7b480c9d50 2024-04-26T20:23:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.444+0000 7f7b4df39640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7b2c06cbc0 0x7f7b2c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.445+0000 7f7b4df39640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7b2c06cbc0 0x7f7b2c06f080 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b44033000 tx=0x7f7b44008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:56.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.448+0000 7f7b3e7fc640 1 -- 172.21.15.37:0/3524886288 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7b3801a310 con 0x7f7b480c9d50 2024-04-26T20:23:56.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.670+0000 7f7b5019d640 1 -- 172.21.15.37:0/3524886288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b48073e70 con 0x7f7b480c9d50 2024-04-26T20:23:56.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.671+0000 7f7b3e7fc640 1 -- 172.21.15.37:0/3524886288 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5181 (secure 0 0 0) 0x7f7b3805a040 con 0x7f7b480c9d50 2024-04-26T20:23:56.672 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:23:56.672 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/4166977617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4166977617},{"type":"v1","addr":"172.21.15.134:6836","nonce":4166977617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3245561282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3245561282},{"type":"v1","addr":"172.21.15.134:6837","nonce":3245561282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3853963851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3853963851},{"type":"v1","addr":"172.21.15.37:6835","nonce":3853963851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:22:42.116910+0000","modified":"2024-04-26T20:22:45.119356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5097":{"gid":5097,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/64661056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":64661056},{"type":"v1","addr":"172.21.15.37:6834","nonce":64661056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5097,"qdb_cluster":[5097]},"id":2}]} 2024-04-26T20:23:56.673 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 16 2024-04-26T20:23:56.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 -- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7b2c06cbc0 msgr2=0x7f7b2c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7b2c06cbc0 0x7f7b2c06f080 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b44033000 tx=0x7f7b44008030 comp rx=0 tx=0).stop 2024-04-26T20:23:56.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 -- 172.21.15.37:0/3524886288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b480c9d50 msgr2=0x7f7b480c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b480c9d50 0x7f7b480c82d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800f7a0 tx=0x7f7b3800fc60 comp rx=0 tx=0).stop 2024-04-26T20:23:56.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 -- 172.21.15.37:0/3524886288 shutdown_connections 2024-04-26T20:23:56.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b480c8810 0x7f7b480af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b480c9d50 0x7f7b480c82d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7b2c06cbc0 0x7f7b2c06f080 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 --2- 172.21.15.37:0/3524886288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34003b70 0x7f7b4809ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 -- 172.21.15.37:0/3524886288 >> 172.21.15.37:0/3524886288 conn(0x7f7b48073880 msgr2=0x7f7b48161af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:56.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 -- 172.21.15.37:0/3524886288 shutdown_connections 2024-04-26T20:23:56.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.673+0000 7f7b5019d640 1 -- 172.21.15.37:0/3524886288 wait complete. 2024-04-26T20:23:56.685 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:23:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.784+0000 7f274a546640 1 Processor -- start 2024-04-26T20:23:56.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.784+0000 7f274a546640 1 -- start start 2024-04-26T20:23:56.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.785+0000 7f274a546640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2744162b40 0x7f27440d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.785+0000 7f274a546640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f27440d3c80 0x7f27440d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.785+0000 7f274a546640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27440d0210 con 0x7f27440d3820 2024-04-26T20:23:56.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.785+0000 7f274a546640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f27440d04f0 con 0x7f2744162b40 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.785+0000 7f274a546640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f27440d07d0 con 0x7f27440d3c80 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.785+0000 7f27437fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f27440d3820 0x7f27440d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33176/0 (socket says 172.21.15.37:33176) 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.785+0000 7f27437fe640 1 -- 172.21.15.37:0/4135638035 learned_addr learned my addr 172.21.15.37:0/4135638035 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.785+0000 7f2743fff640 1 --2- 172.21.15.37:0/4135638035 >> v2:172.21.15.37:3300/0 conn(0x7f2744162b40 0x7f27440d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.786+0000 7f2748ae3640 1 --2- 172.21.15.37:0/4135638035 >> v2:172.21.15.134:3300/0 conn(0x7f27440d3c80 0x7f27440d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.786+0000 7f2743fff640 1 -- 172.21.15.37:0/4135638035 >> v2:172.21.15.134:3300/0 conn(0x7f27440d3c80 msgr2=0x7f27440d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.786+0000 7f2743fff640 1 --2- 172.21.15.37:0/4135638035 >> v2:172.21.15.134:3300/0 conn(0x7f27440d3c80 0x7f27440d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.786+0000 7f2743fff640 1 -- 172.21.15.37:0/4135638035 >> v1:172.21.15.37:6789/0 conn(0x7f27440d3820 legacy=0x7f27440d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.786+0000 7f2743fff640 1 -- 172.21.15.37:0/4135638035 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27440d0ab0 con 0x7f2744162b40 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.786+0000 7f2742ffd640 1 -- 172.21.15.37:0/4135638035 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689079962 0 0) 0x7f27440d0210 con 0x7f27440d3820 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.786+0000 7f2748ae3640 1 --2- 172.21.15.37:0/4135638035 >> v2:172.21.15.134:3300/0 conn(0x7f27440d3c80 0x7f27440d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.786+0000 7f2743fff640 1 --2- 172.21.15.37:0/4135638035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2744162b40 0x7f27440d5bd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f274409ec60 tx=0x7f272c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c2fe99fca91ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.787+0000 7f2742ffd640 1 -- 172.21.15.37:0/4135638035 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c037070 con 0x7f2744162b40 2024-04-26T20:23:56.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.787+0000 7f2742ffd640 1 -- 172.21.15.37:0/4135638035 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c032440 con 0x7f2744162b40 2024-04-26T20:23:56.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.787+0000 7f2742ffd640 1 -- 172.21.15.37:0/4135638035 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c032810 con 0x7f2744162b40 2024-04-26T20:23:56.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.787+0000 7f274a546640 1 -- 172.21.15.37:0/4135638035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2744162b40 msgr2=0x7f27440d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.787+0000 7f274a546640 1 --2- 172.21.15.37:0/4135638035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2744162b40 0x7f27440d5bd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f274409ec60 tx=0x7f272c02a230 comp rx=0 tx=0).stop 2024-04-26T20:23:56.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.787+0000 7f274a546640 1 -- 172.21.15.37:0/4135638035 shutdown_connections 2024-04-26T20:23:56.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.787+0000 7f274a546640 1 --2- 172.21.15.37:0/4135638035 >> v2:172.21.15.134:3300/0 conn(0x7f27440d3c80 0x7f27440d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.787+0000 7f274a546640 1 --2- 172.21.15.37:0/4135638035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2744162b40 0x7f27440d5bd0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.787+0000 7f274a546640 1 -- 172.21.15.37:0/4135638035 >> 172.21.15.37:0/4135638035 conn(0x7f274409e550 msgr2=0x7f27440d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:56.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.787+0000 7f274a546640 1 -- 172.21.15.37:0/4135638035 shutdown_connections 2024-04-26T20:23:56.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.787+0000 7f274a546640 1 -- 172.21.15.37:0/4135638035 wait complete. 2024-04-26T20:23:56.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.788+0000 7f274a546640 1 Processor -- start 2024-04-26T20:23:56.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.789+0000 7f274a546640 1 -- start start 2024-04-26T20:23:56.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.789+0000 7f274a546640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27440d3820 0x7f274414f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.789+0000 7f274a546640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27440d3c80 0x7f27441f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.789+0000 7f274a546640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2744162b40 0x7f27440cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.789+0000 7f274a546640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27440b5bb0 con 0x7f27440d3c80 2024-04-26T20:23:56.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.789+0000 7f274a546640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27440d4110 con 0x7f2744162b40 2024-04-26T20:23:56.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.789+0000 7f274a546640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27440d6110 con 0x7f27440d3820 2024-04-26T20:23:56.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f27437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27440d3c80 0x7f27441f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f27437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27440d3c80 0x7f27441f81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59306/0 (socket says 172.21.15.37:59306) 2024-04-26T20:23:56.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f27437fe640 1 -- 172.21.15.37:0/3401891703 learned_addr learned my addr 172.21.15.37:0/3401891703 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:56.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f2743fff640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27440d3820 0x7f274414f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f2748ae3640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2744162b40 0x7f27440cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f27437fe640 1 -- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27440d3820 msgr2=0x7f274414f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f27437fe640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27440d3820 0x7f274414f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f27437fe640 1 -- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2744162b40 msgr2=0x7f27440cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f27437fe640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2744162b40 0x7f27440cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f27437fe640 1 -- 172.21.15.37:0/3401891703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27441f8710 con 0x7f27440d3c80 2024-04-26T20:23:56.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f27437fe640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27440d3c80 0x7f27441f81d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f273400c430 tx=0x7f273400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:56.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f2740ff9640 1 -- 172.21.15.37:0/3401891703 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340074e0 con 0x7f27440d3c80 2024-04-26T20:23:56.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f2740ff9640 1 -- 172.21.15.37:0/3401891703 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27340047e0 con 0x7f27440d3c80 2024-04-26T20:23:56.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f274a546640 1 -- 172.21.15.37:0/3401891703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27440cd8a0 con 0x7f27440d3c80 2024-04-26T20:23:56.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f2740ff9640 1 -- 172.21.15.37:0/3401891703 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734004bd0 con 0x7f27440d3c80 2024-04-26T20:23:56.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.790+0000 7f274a546640 1 -- 172.21.15.37:0/3401891703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27440cde00 con 0x7f27440d3c80 2024-04-26T20:23:56.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.791+0000 7f274a546640 1 -- 172.21.15.37:0/3401891703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274406a360 con 0x7f27440d3c80 2024-04-26T20:23:56.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.792+0000 7f2740ff9640 1 -- 172.21.15.37:0/3401891703 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f2734011070 con 0x7f27440d3c80 2024-04-26T20:23:56.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.792+0000 7f2740ff9640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f272406cbc0 0x7f272406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:56.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.792+0000 7f2740ff9640 1 -- 172.21.15.37:0/3401891703 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f273408ba40 con 0x7f27440d3c80 2024-04-26T20:23:56.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.792+0000 7f2743fff640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f272406cbc0 0x7f272406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:56.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.793+0000 7f2743fff640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f272406cbc0 0x7f272406f080 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f272c034000 tx=0x7f272c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:56.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.797+0000 7f2740ff9640 1 -- 172.21.15.37:0/3401891703 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f27440cd8a0 con 0x7f27440d3c80 2024-04-26T20:23:56.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.982+0000 7f274a546640 1 -- 172.21.15.37:0/3401891703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27440d2960 con 0x7f27440d3c80 2024-04-26T20:23:56.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.983+0000 7f2740ff9640 1 -- 172.21.15.37:0/3401891703 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16746 (secure 0 0 0) 0x7f2734059540 con 0x7f27440d3c80 2024-04-26T20:23:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:23:56.984 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:23:34.988080+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:22:39.762703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:22:41.066360+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:23:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 -- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f272406cbc0 msgr2=0x7f272406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f272406cbc0 0x7f272406f080 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f272c034000 tx=0x7f272c02e000 comp rx=0 tx=0).stop 2024-04-26T20:23:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 -- 172.21.15.37:0/3401891703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27440d3c80 msgr2=0x7f27441f81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27440d3c80 0x7f27441f81d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f273400c430 tx=0x7f273400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 -- 172.21.15.37:0/3401891703 shutdown_connections 2024-04-26T20:23:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2744162b40 0x7f27440cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27440d3c80 0x7f27441f81d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f272406cbc0 0x7f272406f080 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 --2- 172.21.15.37:0/3401891703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27440d3820 0x7f274414f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 -- 172.21.15.37:0/3401891703 >> 172.21.15.37:0/3401891703 conn(0x7f274409e550 msgr2=0x7f27440b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:56.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 -- 172.21.15.37:0/3401891703 shutdown_connections 2024-04-26T20:23:56.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:56.985+0000 7f274a546640 1 -- 172.21.15.37:0/3401891703 wait complete. 2024-04-26T20:23:56.997 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:23:57.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.098+0000 7f6792047640 1 Processor -- start 2024-04-26T20:23:57.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.098+0000 7f6792047640 1 -- start start 2024-04-26T20:23:57.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.099+0000 7f6792047640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f678c161200 0x7f678c0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.099+0000 7f6792047640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f678c161b80 con 0x7f678c06f790 2024-04-26T20:23:57.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.099+0000 7f6792047640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f678c0b4400 con 0x7f678c160d70 2024-04-26T20:23:57.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.099+0000 7f6792047640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f678c0b4120 con 0x7f678c161200 2024-04-26T20:23:57.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.099+0000 7f678bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f678c161200 0x7f678c0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.099+0000 7f678b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f678c06f790 0x7f678c074030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33192/0 (socket says 172.21.15.37:33192) 2024-04-26T20:23:57.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.099+0000 7f678b7fe640 1 -- 172.21.15.37:0/192300543 learned_addr learned my addr 172.21.15.37:0/192300543 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:57.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.099+0000 7f678bfff640 1 -- 172.21.15.37:0/192300543 >> v1:172.21.15.134:6789/0 conn(0x7f678c160d70 legacy=0x7f678c0b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:57.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.099+0000 7f678bfff640 1 -- 172.21.15.37:0/192300543 >> v1:172.21.15.37:6789/0 conn(0x7f678c06f790 legacy=0x7f678c074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:57.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.099+0000 7f678bfff640 1 -- 172.21.15.37:0/192300543 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c072ff0 con 0x7f678c161200 2024-04-26T20:23:57.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.100+0000 7f678bfff640 1 --2- 172.21.15.37:0/192300543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f678c161200 0x7f678c0b6210 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6784009d90 tx=0x7f678402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3f5ffaac72a404 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:57.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.100+0000 7f678a7fc640 1 -- 172.21.15.37:0/192300543 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784007740 con 0x7f678c161200 2024-04-26T20:23:57.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.100+0000 7f678a7fc640 1 -- 172.21.15.37:0/192300543 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67840078e0 con 0x7f678c161200 2024-04-26T20:23:57.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.100+0000 7f678a7fc640 1 -- 172.21.15.37:0/192300543 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784007cb0 con 0x7f678c161200 2024-04-26T20:23:57.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.101+0000 7f6792047640 1 -- 172.21.15.37:0/192300543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f678c161200 msgr2=0x7f678c0b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:57.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.101+0000 7f6792047640 1 --2- 172.21.15.37:0/192300543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f678c161200 0x7f678c0b6210 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6784009d90 tx=0x7f678402a6a0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.101+0000 7f6792047640 1 -- 172.21.15.37:0/192300543 shutdown_connections 2024-04-26T20:23:57.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.101+0000 7f6792047640 1 --2- 172.21.15.37:0/192300543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f678c161200 0x7f678c0b6210 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.101+0000 7f6792047640 1 -- 172.21.15.37:0/192300543 >> 172.21.15.37:0/192300543 conn(0x7f678c0d0d50 msgr2=0x7f678c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:57.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.101+0000 7f6792047640 1 -- 172.21.15.37:0/192300543 shutdown_connections 2024-04-26T20:23:57.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.101+0000 7f6792047640 1 -- 172.21.15.37:0/192300543 wait complete. 2024-04-26T20:23:57.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.102+0000 7f6792047640 1 Processor -- start 2024-04-26T20:23:57.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.102+0000 7f6792047640 1 -- start start 2024-04-26T20:23:57.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f6792047640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f678c06f790 0x7f678c1e91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f6792047640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f678c160d70 0x7f678c156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f6792047640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f678c161200 0x7f678c1e7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f6792047640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f678c161650 con 0x7f678c161200 2024-04-26T20:23:57.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f6792047640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f678c0735b0 con 0x7f678c160d70 2024-04-26T20:23:57.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f6792047640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f678c0b3d80 con 0x7f678c06f790 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f678bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f678c161200 0x7f678c1e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f678bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f678c161200 0x7f678c1e7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59334/0 (socket says 172.21.15.37:59334) 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f678bfff640 1 -- 172.21.15.37:0/2139300708 learned_addr learned my addr 172.21.15.37:0/2139300708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f678affd640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f678c160d70 0x7f678c156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f678b7fe640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f678c06f790 0x7f678c1e91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f678bfff640 1 -- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f678c06f790 msgr2=0x7f678c1e91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f678bfff640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f678c06f790 0x7f678c1e91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f678bfff640 1 -- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f678c160d70 msgr2=0x7f678c156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f678bfff640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f678c160d70 0x7f678c156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f678bfff640 1 -- 172.21.15.37:0/2139300708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c1e9810 con 0x7f678c161200 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.103+0000 7f678bfff640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f678c161200 0x7f678c1e7650 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6784009d90 tx=0x7f6784004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.104+0000 7f6773fff640 1 -- 172.21.15.37:0/2139300708 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784040070 con 0x7f678c161200 2024-04-26T20:23:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.104+0000 7f6773fff640 1 -- 172.21.15.37:0/2139300708 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6784031c90 con 0x7f678c161200 2024-04-26T20:23:57.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.104+0000 7f6792047640 1 -- 172.21.15.37:0/2139300708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678c1e9af0 con 0x7f678c161200 2024-04-26T20:23:57.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.104+0000 7f6773fff640 1 -- 172.21.15.37:0/2139300708 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678403c640 con 0x7f678c161200 2024-04-26T20:23:57.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.104+0000 7f6792047640 1 -- 172.21.15.37:0/2139300708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678c1e9dd0 con 0x7f678c161200 2024-04-26T20:23:57.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.105+0000 7f6792047640 1 -- 172.21.15.37:0/2139300708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678c06a360 con 0x7f678c161200 2024-04-26T20:23:57.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.105+0000 7f6773fff640 1 -- 172.21.15.37:0/2139300708 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f6784039040 con 0x7f678c161200 2024-04-26T20:23:57.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.106+0000 7f6773fff640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f677806cbc0 0x7f677806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.106+0000 7f6773fff640 1 -- 172.21.15.37:0/2139300708 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5737+0+0 (secure 0 0 0) 0x7f67840ad800 con 0x7f678c161200 2024-04-26T20:23:57.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.106+0000 7f678b7fe640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f677806cbc0 0x7f677806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.106+0000 7f678b7fe640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f677806cbc0 0x7f677806f080 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f677c000f60 tx=0x7f677c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.110+0000 7f6773fff640 1 -- 172.21.15.37:0/2139300708 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f678407b7e0 con 0x7f678c161200 2024-04-26T20:23:57.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.332+0000 7f6792047640 1 -- 172.21.15.37:0/2139300708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f678c15d1c0 con 0x7f678c161200 2024-04-26T20:23:57.334 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-26T20:23:57.333+0000 7f284b532640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:23:57.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.336+0000 7f6773fff640 1 -- 172.21.15.37:0/2139300708 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f678407b0e0 con 0x7f678c161200 2024-04-26T20:23:57.337 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:23:57.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 -- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f677806cbc0 msgr2=0x7f677806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f677806cbc0 0x7f677806f080 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f677c000f60 tx=0x7f677c009040 comp rx=0 tx=0).stop 2024-04-26T20:23:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 -- 172.21.15.37:0/2139300708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f678c161200 msgr2=0x7f678c1e7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f678c161200 0x7f678c1e7650 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6784009d90 tx=0x7f6784004a40 comp rx=0 tx=0).stop 2024-04-26T20:23:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 -- 172.21.15.37:0/2139300708 shutdown_connections 2024-04-26T20:23:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f678c161200 0x7f678c1e7650 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f678c160d70 0x7f678c156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f677806cbc0 0x7f677806f080 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 --2- 172.21.15.37:0/2139300708 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f678c06f790 0x7f678c1e91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 -- 172.21.15.37:0/2139300708 >> 172.21.15.37:0/2139300708 conn(0x7f678c0d0d50 msgr2=0x7f678c0d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 -- 172.21.15.37:0/2139300708 shutdown_connections 2024-04-26T20:23:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.339+0000 7f6792047640 1 -- 172.21.15.37:0/2139300708 wait complete. 2024-04-26T20:23:57.352 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:23:57.377 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-26T20:23:57.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.456+0000 7f09b8ff0640 1 Processor -- start 2024-04-26T20:23:57.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.456+0000 7f09b8ff0640 1 -- start start 2024-04-26T20:23:57.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09b8ff0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b40d15d0 0x7f09b40d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09b8ff0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09b415d630 0x7f09b40cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09b8ff0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f09b40d0dd0 0x7f09b40cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09b8ff0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f09b40cf500 con 0x7f09b415d630 2024-04-26T20:23:57.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09b8ff0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f09b40d09d0 con 0x7f09b40d0dd0 2024-04-26T20:23:57.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09b8ff0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09b40d1ef0 con 0x7f09b40d15d0 2024-04-26T20:23:57.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09abfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09b415d630 0x7f09b40cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09abfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09b415d630 0x7f09b40cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59374/0 (socket says 172.21.15.37:59374) 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09abfff640 1 -- 172.21.15.37:0/1269394980 learned_addr learned my addr 172.21.15.37:0/1269394980 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09b259c640 1 --2- 172.21.15.37:0/1269394980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b40d15d0 0x7f09b40d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09b2d9d640 1 --2- 172.21.15.37:0/1269394980 >> v2:172.21.15.134:3300/0 conn(0x7f09b40d0dd0 0x7f09b40cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09abfff640 1 -- 172.21.15.37:0/1269394980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b40d15d0 msgr2=0x7f09b40d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09abfff640 1 --2- 172.21.15.37:0/1269394980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b40d15d0 0x7f09b40d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09abfff640 1 -- 172.21.15.37:0/1269394980 >> v2:172.21.15.134:3300/0 conn(0x7f09b40d0dd0 msgr2=0x7f09b40cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09abfff640 1 --2- 172.21.15.37:0/1269394980 >> v2:172.21.15.134:3300/0 conn(0x7f09b40d0dd0 0x7f09b40cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09abfff640 1 -- 172.21.15.37:0/1269394980 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b40cec60 con 0x7f09b415d630 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09b259c640 1 --2- 172.21.15.37:0/1269394980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b40d15d0 0x7f09b40d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.457+0000 7f09b2d9d640 1 --2- 172.21.15.37:0/1269394980 >> v2:172.21.15.134:3300/0 conn(0x7f09b40d0dd0 0x7f09b40cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.458+0000 7f09abfff640 1 --2- 172.21.15.37:0/1269394980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b415d630 0x7f09b40cf870 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f09b406fc90 tx=0x7f099c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a324b575717a075a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.458+0000 7f09b1d9b640 1 -- 172.21.15.37:0/1269394980 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c037070 con 0x7f09b415d630 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.458+0000 7f09b1d9b640 1 -- 172.21.15.37:0/1269394980 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f099c032440 con 0x7f09b415d630 2024-04-26T20:23:57.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.458+0000 7f09b1d9b640 1 -- 172.21.15.37:0/1269394980 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c032810 con 0x7f09b415d630 2024-04-26T20:23:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.459+0000 7f09b8ff0640 1 -- 172.21.15.37:0/1269394980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b415d630 msgr2=0x7f09b40cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.459+0000 7f09b8ff0640 1 --2- 172.21.15.37:0/1269394980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b415d630 0x7f09b40cf870 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f09b406fc90 tx=0x7f099c02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.459+0000 7f09b8ff0640 1 -- 172.21.15.37:0/1269394980 shutdown_connections 2024-04-26T20:23:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.459+0000 7f09b8ff0640 1 --2- 172.21.15.37:0/1269394980 >> v2:172.21.15.134:3300/0 conn(0x7f09b40d0dd0 0x7f09b40cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.459+0000 7f09b8ff0640 1 --2- 172.21.15.37:0/1269394980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b415d630 0x7f09b40cf870 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.459+0000 7f09b8ff0640 1 --2- 172.21.15.37:0/1269394980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b40d15d0 0x7f09b40d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.459+0000 7f09b8ff0640 1 -- 172.21.15.37:0/1269394980 >> 172.21.15.37:0/1269394980 conn(0x7f09b40d2ff0 msgr2=0x7f09b40d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.459+0000 7f09b8ff0640 1 -- 172.21.15.37:0/1269394980 shutdown_connections 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.459+0000 7f09b8ff0640 1 -- 172.21.15.37:0/1269394980 wait complete. 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.460+0000 7f09b8ff0640 1 Processor -- start 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.460+0000 7f09b8ff0640 1 -- start start 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b8ff0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b40d0dd0 0x7f09b40cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b8ff0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b40d15d0 0x7f09b40b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b8ff0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09b415d630 0x7f09b40b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b8ff0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09b40b2570 con 0x7f09b40d0dd0 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b8ff0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09b40cf220 con 0x7f09b415d630 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b8ff0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09b40ce8f0 con 0x7f09b40d15d0 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b40d0dd0 0x7f09b40cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b40d0dd0 0x7f09b40cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59382/0 (socket says 172.21.15.37:59382) 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b259c640 1 -- 172.21.15.37:0/2790879506 learned_addr learned my addr 172.21.15.37:0/2790879506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09abfff640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b40d15d0 0x7f09b40b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b2d9d640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09b415d630 0x7f09b40b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b259c640 1 -- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b40d15d0 msgr2=0x7f09b40b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b259c640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b40d15d0 0x7f09b40b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b259c640 1 -- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09b415d630 msgr2=0x7f09b40b7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b259c640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09b415d630 0x7f09b40b7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b259c640 1 -- 172.21.15.37:0/2790879506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b4155890 con 0x7f09b40d0dd0 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.461+0000 7f09b259c640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b40d0dd0 0x7f09b40cc8f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c820 tx=0x7f09a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.462+0000 7f09aaffd640 1 -- 172.21.15.37:0/2790879506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4015070 con 0x7f09b40d0dd0 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.462+0000 7f09aaffd640 1 -- 172.21.15.37:0/2790879506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a4007a30 con 0x7f09b40d0dd0 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.462+0000 7f09b8ff0640 1 -- 172.21.15.37:0/2790879506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b40c92a0 con 0x7f09b40d0dd0 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.462+0000 7f09aaffd640 1 -- 172.21.15.37:0/2790879506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4007de0 con 0x7f09b40d0dd0 2024-04-26T20:23:57.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.462+0000 7f09b8ff0640 1 -- 172.21.15.37:0/2790879506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b40c8910 con 0x7f09b40d0dd0 2024-04-26T20:23:57.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.463+0000 7f09b8ff0640 1 -- 172.21.15.37:0/2790879506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b406a360 con 0x7f09b40d0dd0 2024-04-26T20:23:57.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.463+0000 7f09aaffd640 1 -- 172.21.15.37:0/2790879506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f09a4018070 con 0x7f09b40d0dd0 2024-04-26T20:23:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.463+0000 7f09aaffd640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f099406cbc0 0x7f099406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.464+0000 7f09aaffd640 1 -- 172.21.15.37:0/2790879506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f09a408bf10 con 0x7f09b40d0dd0 2024-04-26T20:23:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.464+0000 7f09abfff640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f099406cbc0 0x7f099406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:57.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.464+0000 7f09abfff640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f099406cbc0 0x7f099406f080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f099c034000 tx=0x7f099c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:57.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.468+0000 7f09aaffd640 1 -- 172.21.15.37:0/2790879506 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f09b4155890 con 0x7f09b40d0dd0 2024-04-26T20:23:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:57.691+0000 7f09b8ff0640 1 -- 172.21.15.37:0/2790879506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f09b40744b0 con 0x7f09b40d0dd0 2024-04-26T20:23:58.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.340+0000 7f09aaffd640 1 -- 172.21.15.37:0/2790879506 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f09a40598c0 con 0x7f09b40d0dd0 2024-04-26T20:23:58.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 -- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f099406cbc0 msgr2=0x7f099406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:58.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f099406cbc0 0x7f099406f080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f099c034000 tx=0x7f099c02e000 comp rx=0 tx=0).stop 2024-04-26T20:23:58.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 -- 172.21.15.37:0/2790879506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b40d0dd0 msgr2=0x7f09b40cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:58.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b40d0dd0 0x7f09b40cc8f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c820 tx=0x7f09a400cce0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 -- 172.21.15.37:0/2790879506 shutdown_connections 2024-04-26T20:23:58.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09b415d630 0x7f09b40b7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f099406cbc0 0x7f099406f080 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b40d15d0 0x7f09b40b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 --2- 172.21.15.37:0/2790879506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b40d0dd0 0x7f09b40cc8f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 -- 172.21.15.37:0/2790879506 >> 172.21.15.37:0/2790879506 conn(0x7f09b40d2ff0 msgr2=0x7f09b4071ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:58.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 -- 172.21.15.37:0/2790879506 shutdown_connections 2024-04-26T20:23:58.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.343+0000 7f09b8ff0640 1 -- 172.21.15.37:0/2790879506 wait complete. 2024-04-26T20:23:58.357 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:23:58.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.507+0000 7f7df6c2a640 1 Processor -- start 2024-04-26T20:23:58.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.507+0000 7f7df6c2a640 1 -- start start 2024-04-26T20:23:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df6c2a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7df015f390 0x7f7df0072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df6c2a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df00b1e50 0x7f7df00cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df6c2a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7df006e9c0 0x7f7df006ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df6c2a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f7df00d2db0 con 0x7f7df015f390 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df6c2a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7df015f770 con 0x7f7df006e9c0 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df6c2a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7df00d2ad0 con 0x7f7df00b1e50 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df49c6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7df015f390 0x7f7df0072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df49c6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7df015f390 0x7f7df0072620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59394/0 (socket says 172.21.15.37:59394) 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df49c6640 1 -- 172.21.15.37:0/3406825581 learned_addr learned my addr 172.21.15.37:0/3406825581 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7deffff640 1 --2- 172.21.15.37:0/3406825581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df00b1e50 0x7f7df00cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df51c7640 1 --2- 172.21.15.37:0/3406825581 >> v2:172.21.15.134:3300/0 conn(0x7f7df006e9c0 0x7f7df006ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df49c6640 1 -- 172.21.15.37:0/3406825581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df00b1e50 msgr2=0x7f7df00cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df49c6640 1 --2- 172.21.15.37:0/3406825581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df00b1e50 0x7f7df00cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df49c6640 1 -- 172.21.15.37:0/3406825581 >> v2:172.21.15.134:3300/0 conn(0x7f7df006e9c0 msgr2=0x7f7df006ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df49c6640 1 --2- 172.21.15.37:0/3406825581 >> v2:172.21.15.134:3300/0 conn(0x7f7df006e9c0 0x7f7df006ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.508+0000 7f7df49c6640 1 -- 172.21.15.37:0/3406825581 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df00d0c60 con 0x7f7df015f390 2024-04-26T20:23:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.509+0000 7f7df49c6640 1 --2- 172.21.15.37:0/3406825581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7df015f390 0x7f7df0072620 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7df00b2990 tx=0x7f7de402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f872490a4dbdc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.509+0000 7f7def7fe640 1 -- 172.21.15.37:0/3406825581 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4037070 con 0x7f7df015f390 2024-04-26T20:23:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.509+0000 7f7def7fe640 1 -- 172.21.15.37:0/3406825581 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4032440 con 0x7f7df015f390 2024-04-26T20:23:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.509+0000 7f7def7fe640 1 -- 172.21.15.37:0/3406825581 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4032810 con 0x7f7df015f390 2024-04-26T20:23:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.510+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3406825581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7df015f390 msgr2=0x7f7df0072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.510+0000 7f7df6c2a640 1 --2- 172.21.15.37:0/3406825581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7df015f390 0x7f7df0072620 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7df00b2990 tx=0x7f7de402a140 comp rx=0 tx=0).stop 2024-04-26T20:23:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.510+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3406825581 shutdown_connections 2024-04-26T20:23:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.510+0000 7f7df6c2a640 1 --2- 172.21.15.37:0/3406825581 >> v2:172.21.15.134:3300/0 conn(0x7f7df006e9c0 0x7f7df006ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.510+0000 7f7df6c2a640 1 --2- 172.21.15.37:0/3406825581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df00b1e50 0x7f7df00cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.510+0000 7f7df6c2a640 1 --2- 172.21.15.37:0/3406825581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7df015f390 0x7f7df0072620 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.510+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3406825581 >> 172.21.15.37:0/3406825581 conn(0x7f7df00d2f60 msgr2=0x7f7df00d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.510+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3406825581 shutdown_connections 2024-04-26T20:23:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.510+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3406825581 wait complete. 2024-04-26T20:23:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.511+0000 7f7df6c2a640 1 Processor -- start 2024-04-26T20:23:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.511+0000 7f7df6c2a640 1 -- start start 2024-04-26T20:23:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df6c2a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df006e9c0 0x7f7df00b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df6c2a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df00b1e50 0x7f7df01f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df6c2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7df015f390 0x7f7df01f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df6c2a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00d1220 con 0x7f7df015f390 2024-04-26T20:23:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df6c2a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00d2780 con 0x7f7df00b1e50 2024-04-26T20:23:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df6c2a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7df01631c0 con 0x7f7df006e9c0 2024-04-26T20:23:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df51c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7df015f390 0x7f7df01f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df51c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7df015f390 0x7f7df01f8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59410/0 (socket says 172.21.15.37:59410) 2024-04-26T20:23:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df51c7640 1 -- 172.21.15.37:0/3279143758 learned_addr learned my addr 172.21.15.37:0/3279143758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df49c6640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df006e9c0 0x7f7df00b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7deffff640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df00b1e50 0x7f7df01f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df51c7640 1 -- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df006e9c0 msgr2=0x7f7df00b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df51c7640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df006e9c0 0x7f7df00b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df51c7640 1 -- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df00b1e50 msgr2=0x7f7df01f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df51c7640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df00b1e50 0x7f7df01f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df51c7640 1 -- 172.21.15.37:0/3279143758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df01f5300 con 0x7f7df015f390 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.512+0000 7f7df51c7640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7df015f390 0x7f7df01f8a50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7de8007e70 tx=0x7f7de800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.513+0000 7f7ded7fa640 1 -- 172.21.15.37:0/3279143758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8014620 con 0x7f7df015f390 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.513+0000 7f7ded7fa640 1 -- 172.21.15.37:0/3279143758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de8019070 con 0x7f7df015f390 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.513+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3279143758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df00b03d0 con 0x7f7df015f390 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.513+0000 7f7ded7fa640 1 -- 172.21.15.37:0/3279143758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8014a20 con 0x7f7df015f390 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.513+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3279143758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df00b0670 con 0x7f7df015f390 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.514+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3279143758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df006a360 con 0x7f7df015f390 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.514+0000 7f7ded7fa640 1 -- 172.21.15.37:0/3279143758 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f7de8014bc0 con 0x7f7df015f390 2024-04-26T20:23:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.515+0000 7f7ded7fa640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7dd406cbc0 0x7f7dd406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.515+0000 7f7ded7fa640 1 -- 172.21.15.37:0/3279143758 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f7de808c820 con 0x7f7df015f390 2024-04-26T20:23:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.515+0000 7f7df49c6640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7dd406cbc0 0x7f7dd406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.515+0000 7f7df49c6640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7dd406cbc0 0x7f7dd406f080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7de4034000 tx=0x7f7de402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.519+0000 7f7ded7fa640 1 -- 172.21.15.37:0/3279143758 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7de805a250 con 0x7f7df015f390 2024-04-26T20:23:58.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:58.705+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3279143758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7df00b8ee0 con 0x7f7df015f390 2024-04-26T20:23:59.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.342+0000 7f7ded7fa640 1 -- 172.21.15.37:0/3279143758 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7f7de805e550 con 0x7f7df015f390 2024-04-26T20:23:59.343 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:23:59.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.344+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7dd406cbc0 msgr2=0x7f7dd406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:59.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.344+0000 7f7df6c2a640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7dd406cbc0 0x7f7dd406f080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7de4034000 tx=0x7f7de402e000 comp rx=0 tx=0).stop 2024-04-26T20:23:59.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.344+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3279143758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7df015f390 msgr2=0x7f7df01f8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:59.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.344+0000 7f7df6c2a640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7df015f390 0x7f7df01f8a50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7de8007e70 tx=0x7f7de800e4b0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.344+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3279143758 shutdown_connections 2024-04-26T20:23:59.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.344+0000 7f7df6c2a640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7df015f390 0x7f7df01f8a50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.344+0000 7f7df6c2a640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df00b1e50 0x7f7df01f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.344+0000 7f7df6c2a640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7dd406cbc0 0x7f7dd406f080 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.344+0000 7f7df6c2a640 1 --2- 172.21.15.37:0/3279143758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df006e9c0 0x7f7df00b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.344+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3279143758 >> 172.21.15.37:0/3279143758 conn(0x7f7df00d2f60 msgr2=0x7f7df0160370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:59.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.345+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3279143758 shutdown_connections 2024-04-26T20:23:59.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.345+0000 7f7df6c2a640 1 -- 172.21.15.37:0/3279143758 wait complete. 2024-04-26T20:23:59.359 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:23:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.467+0000 7f7f82e74640 1 Processor -- start 2024-04-26T20:23:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.468+0000 7f7f82e74640 1 -- start start 2024-04-26T20:23:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.468+0000 7f7f82e74640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7f7c0d4f30 0x7f7f7c0d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.468+0000 7f7f82e74640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f7c0b58b0 0x7f7f7c0b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.468+0000 7f7f82e74640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f7c0b50f0 con 0x7f7f7c15f5d0 2024-04-26T20:23:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.468+0000 7f7f82e74640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f7f7c0b5420 con 0x7f7f7c0d4f30 2024-04-26T20:23:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.468+0000 7f7f82e74640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c0d5900 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.469+0000 7f7f7bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7f7c0d4f30 0x7f7f7c0d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.469+0000 7f7f80c10640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7f7c15f5d0 0x7f7f7c0d3ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33208/0 (socket says 172.21.15.37:33208) 2024-04-26T20:23:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.469+0000 7f7f80c10640 1 -- 172.21.15.37:0/989724523 learned_addr learned my addr 172.21.15.37:0/989724523 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:23:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.469+0000 7f7f81411640 1 --2- 172.21.15.37:0/989724523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f7c0b58b0 0x7f7f7c0b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.469+0000 7f7f7bfff640 1 -- 172.21.15.37:0/989724523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f7c0b58b0 msgr2=0x7f7f7c0b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.469+0000 7f7f7bfff640 1 --2- 172.21.15.37:0/989724523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f7c0b58b0 0x7f7f7c0b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.469+0000 7f7f7bfff640 1 -- 172.21.15.37:0/989724523 >> v1:172.21.15.37:6789/0 conn(0x7f7f7c15f5d0 legacy=0x7f7f7c0d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.469+0000 7f7f7bfff640 1 -- 172.21.15.37:0/989724523 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c0d0a80 con 0x7f7f7c0d4f30 2024-04-26T20:23:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f7bfff640 1 --2- 172.21.15.37:0/989724523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c0d4f30 0x7f7f7c0d53c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7f700097d0 tx=0x7f7f7002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1414f26186164aef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f7b7fe640 1 -- 172.21.15.37:0/989724523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70037070 con 0x7f7f7c0d4f30 2024-04-26T20:23:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f7b7fe640 1 -- 172.21.15.37:0/989724523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f70032440 con 0x7f7f7c0d4f30 2024-04-26T20:23:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f7b7fe640 1 -- 172.21.15.37:0/989724523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70032810 con 0x7f7f7c0d4f30 2024-04-26T20:23:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f82e74640 1 -- 172.21.15.37:0/989724523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c0d4f30 msgr2=0x7f7f7c0d53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f82e74640 1 --2- 172.21.15.37:0/989724523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c0d4f30 0x7f7f7c0d53c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7f700097d0 tx=0x7f7f7002a250 comp rx=0 tx=0).stop 2024-04-26T20:23:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f82e74640 1 -- 172.21.15.37:0/989724523 shutdown_connections 2024-04-26T20:23:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f82e74640 1 --2- 172.21.15.37:0/989724523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f7c0b58b0 0x7f7f7c0b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f82e74640 1 --2- 172.21.15.37:0/989724523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c0d4f30 0x7f7f7c0d53c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f82e74640 1 -- 172.21.15.37:0/989724523 >> 172.21.15.37:0/989724523 conn(0x7f7f7c08ebd0 msgr2=0x7f7f7c09e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f82e74640 1 -- 172.21.15.37:0/989724523 shutdown_connections 2024-04-26T20:23:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.470+0000 7f7f82e74640 1 -- 172.21.15.37:0/989724523 wait complete. 2024-04-26T20:23:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.471+0000 7f7f82e74640 1 Processor -- start 2024-04-26T20:23:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.471+0000 7f7f82e74640 1 -- start start 2024-04-26T20:23:59.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.472+0000 7f7f82e74640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c0b58b0 0x7f7f7c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:59.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.472+0000 7f7f82e74640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f7c0d4f30 0x7f7f7c151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:59.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.472+0000 7f7f82e74640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f7c15f5d0 0x7f7f7c151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:59.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.472+0000 7f7f82e74640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c15f9b0 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.472+0000 7f7f82e74640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0d1040 con 0x7f7f7c0d4f30 2024-04-26T20:23:59.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.472+0000 7f7f82e74640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c0d41e0 con 0x7f7f7c15f5d0 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.472+0000 7f7f80c10640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c0b58b0 0x7f7f7c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.472+0000 7f7f80c10640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c0b58b0 0x7f7f7c15e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59424/0 (socket says 172.21.15.37:59424) 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.472+0000 7f7f80c10640 1 -- 172.21.15.37:0/612661359 learned_addr learned my addr 172.21.15.37:0/612661359 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f7bfff640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f7c0d4f30 0x7f7f7c151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f81411640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f7c15f5d0 0x7f7f7c151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f80c10640 1 -- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f7c15f5d0 msgr2=0x7f7f7c151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f80c10640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f7c15f5d0 0x7f7f7c151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f80c10640 1 -- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f7c0d4f30 msgr2=0x7f7f7c151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f80c10640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f7c0d4f30 0x7f7f7c151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f80c10640 1 -- 172.21.15.37:0/612661359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c0b9ef0 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f7bfff640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f7c0d4f30 0x7f7f7c151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f80c10640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c0b58b0 0x7f7f7c15e3b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400ca70 tx=0x7f7f7400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f797fa640 1 -- 172.21.15.37:0/612661359 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74017070 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f797fa640 1 -- 172.21.15.37:0/612661359 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7400abd0 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f82e74640 1 -- 172.21.15.37:0/612661359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7c0ba1d0 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f797fa640 1 -- 172.21.15.37:0/612661359 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74012400 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.473+0000 7f7f82e74640 1 -- 172.21.15.37:0/612661359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f7c0ba790 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.474+0000 7f7f82e74640 1 -- 172.21.15.37:0/612661359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f7c06a360 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.475+0000 7f7f797fa640 1 -- 172.21.15.37:0/612661359 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f7f7401a070 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.475+0000 7f7f797fa640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7f6806cbc0 0x7f7f6806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:59.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.475+0000 7f7f797fa640 1 -- 172.21.15.37:0/612661359 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5357+0+0 (secure 0 0 0) 0x7f7f7408bd00 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.475+0000 7f7f7bfff640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7f6806cbc0 0x7f7f6806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:59.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.476+0000 7f7f7bfff640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7f6806cbc0 0x7f7f6806f080 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7f70034000 tx=0x7f7f7002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:59.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.479+0000 7f7f797fa640 1 -- 172.21.15.37:0/612661359 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7f74059de0 con 0x7f7f7c0b58b0 2024-04-26T20:23:59.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:23:59.674+0000 7f7f82e74640 1 -- 172.21.15.37:0/612661359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7f7c075740 con 0x7f7f7c0b58b0 2024-04-26T20:24:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.346+0000 7f7f797fa640 1 -- 172.21.15.37:0/612661359 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7f7f740596e0 con 0x7f7f7c0b58b0 2024-04-26T20:24:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-26T20:24:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.349+0000 7f7f82e74640 1 -- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7f6806cbc0 msgr2=0x7f7f6806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.349+0000 7f7f82e74640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7f6806cbc0 0x7f7f6806f080 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7f70034000 tx=0x7f7f7002e000 comp rx=0 tx=0).stop 2024-04-26T20:24:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.349+0000 7f7f82e74640 1 -- 172.21.15.37:0/612661359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c0b58b0 msgr2=0x7f7f7c15e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.349+0000 7f7f82e74640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c0b58b0 0x7f7f7c15e3b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400ca70 tx=0x7f7f7400cf30 comp rx=0 tx=0).stop 2024-04-26T20:24:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.350+0000 7f7f82e74640 1 -- 172.21.15.37:0/612661359 shutdown_connections 2024-04-26T20:24:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.350+0000 7f7f82e74640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f7f6806cbc0 0x7f7f6806f080 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.350+0000 7f7f82e74640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f7c15f5d0 0x7f7f7c151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.350+0000 7f7f82e74640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f7c0d4f30 0x7f7f7c151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.350+0000 7f7f82e74640 1 --2- 172.21.15.37:0/612661359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c0b58b0 0x7f7f7c15e3b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.350+0000 7f7f82e74640 1 -- 172.21.15.37:0/612661359 >> 172.21.15.37:0/612661359 conn(0x7f7f7c08ebd0 msgr2=0x7f7f7c0b72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.350+0000 7f7f82e74640 1 -- 172.21.15.37:0/612661359 shutdown_connections 2024-04-26T20:24:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.350+0000 7f7f82e74640 1 -- 172.21.15.37:0/612661359 wait complete. 2024-04-26T20:24:00.366 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_delay 2024-04-26T20:24:00.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.474+0000 7ff0369c9640 1 Processor -- start 2024-04-26T20:24:00.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.474+0000 7ff0369c9640 1 -- start start 2024-04-26T20:24:00.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.475+0000 7ff0369c9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff0300d1c90 0x7ff0300d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.475+0000 7ff0369c9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff03015f940 con 0x7ff03015f560 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.475+0000 7ff0369c9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0300d1270 con 0x7ff0301673b0 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.475+0000 7ff0369c9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff0300d2650 con 0x7ff0300d1c90 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.475+0000 7ff02ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff03015f560 0x7ff0300d0b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33224/0 (socket says 172.21.15.37:33224) 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.475+0000 7ff02ffff640 1 -- 172.21.15.37:0/3317253941 learned_addr learned my addr 172.21.15.37:0/3317253941 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3149741509 0 0) 0x7ff03015f940 con 0x7ff03015f560 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02f7fe640 1 --2- 172.21.15.37:0/3317253941 >> v2:172.21.15.134:3300/0 conn(0x7ff0300d1c90 0x7ff0300d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff018003520 con 0x7ff03015f560 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2936407443 0 0) 0x7ff018003520 con 0x7ff03015f560 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 >> v2:172.21.15.134:3300/0 conn(0x7ff0300d1c90 msgr2=0x7ff0300d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 --2- 172.21.15.37:0/3317253941 >> v2:172.21.15.134:3300/0 conn(0x7ff0300d1c90 0x7ff0300d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 >> v1:172.21.15.134:6789/0 conn(0x7ff0301673b0 legacy=0x7ff0301697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0300d43a0 con 0x7ff03015f560 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7ff020001690 con 0x7ff03015f560 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 >> v1:172.21.15.37:6789/0 conn(0x7ff03015f560 legacy=0x7ff0300d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0180039b0 0x7ff018003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff018004800 0x7ff018004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0180051c0 0x7ff01800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff01800c030 con 0x7ff0180039b0 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff01800c1e0 con 0x7ff0180051c0 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff01800c3e0 con 0x7ff018004800 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02ffff640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0180039b0 0x7ff018003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff034f66640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0180051c0 0x7ff01800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.476+0000 7ff02f7fe640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff018004800 0x7ff018004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.477+0000 7ff02ffff640 1 -- 172.21.15.37:0/3317253941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff018004800 msgr2=0x7ff018004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.477+0000 7ff02ffff640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff018004800 0x7ff018004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.477+0000 7ff02ffff640 1 -- 172.21.15.37:0/3317253941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0180051c0 msgr2=0x7ff01800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.477+0000 7ff02ffff640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0180051c0 0x7ff01800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.477+0000 7ff02ffff640 1 -- 172.21.15.37:0/3317253941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff018003520 con 0x7ff0180039b0 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.477+0000 7ff034f66640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0180051c0 0x7ff01800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.477+0000 7ff034f66640 1 -- 172.21.15.37:0/3317253941 reap_dead start 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.477+0000 7ff02ffff640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0180039b0 0x7ff018003dd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff020007fd0 tx=0x7ff018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7fd0cdf8cd8f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.477+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020007d50 con 0x7ff0180039b0 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.477+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0200058d0 con 0x7ff0180039b0 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.477+0000 7ff02effd640 1 -- 172.21.15.37:0/3317253941 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020005ce0 con 0x7ff0180039b0 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.478+0000 7ff0369c9640 1 -- 172.21.15.37:0/3317253941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0180039b0 msgr2=0x7ff018003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.478+0000 7ff0369c9640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0180039b0 0x7ff018003dd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff020007fd0 tx=0x7ff018005830 comp rx=0 tx=0).stop 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.478+0000 7ff0369c9640 1 -- 172.21.15.37:0/3317253941 shutdown_connections 2024-04-26T20:24:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.478+0000 7ff0369c9640 1 --2- 172.21.15.37:0/3317253941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0180039b0 0x7ff018003dd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.478+0000 7ff0369c9640 1 -- 172.21.15.37:0/3317253941 >> 172.21.15.37:0/3317253941 conn(0x7ff03009e570 msgr2=0x7ff0300d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:00.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.478+0000 7ff0369c9640 1 -- 172.21.15.37:0/3317253941 shutdown_connections 2024-04-26T20:24:00.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.478+0000 7ff0369c9640 1 -- 172.21.15.37:0/3317253941 wait complete. 2024-04-26T20:24:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.479+0000 7ff0369c9640 1 Processor -- start 2024-04-26T20:24:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.479+0000 7ff0369c9640 1 -- start start 2024-04-26T20:24:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff0369c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0180039b0 0x7ff030156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff0369c9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0300c9ba0 0x7ff0300caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff0369c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0300caf50 0x7ff0300b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff0369c9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0300d38c0 con 0x7ff0300c9ba0 2024-04-26T20:24:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff0369c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0300d35e0 con 0x7ff0180039b0 2024-04-26T20:24:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff0369c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff0300d3ba0 con 0x7ff0300caf50 2024-04-26T20:24:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0300c9ba0 0x7ff0300caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0300c9ba0 0x7ff0300caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59440/0 (socket says 172.21.15.37:59440) 2024-04-26T20:24:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff02f7fe640 1 -- 172.21.15.37:0/1723062696 learned_addr learned my addr 172.21.15.37:0/1723062696 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff034f66640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0300caf50 0x7ff0300b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff02ffff640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0180039b0 0x7ff030156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff02f7fe640 1 -- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0300caf50 msgr2=0x7ff0300b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff02f7fe640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0300caf50 0x7ff0300b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff02f7fe640 1 -- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0180039b0 msgr2=0x7ff030156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff02f7fe640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0180039b0 0x7ff030156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.480+0000 7ff02f7fe640 1 -- 172.21.15.37:0/1723062696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0300c8de0 con 0x7ff0300c9ba0 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.481+0000 7ff02f7fe640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0300c9ba0 0x7ff0300caa10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff024011ad0 tx=0x7ff024011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.481+0000 7ff02cff9640 1 -- 172.21.15.37:0/1723062696 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400fb50 con 0x7ff0300c9ba0 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.481+0000 7ff02cff9640 1 -- 172.21.15.37:0/1723062696 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02400fcf0 con 0x7ff0300c9ba0 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.481+0000 7ff0369c9640 1 -- 172.21.15.37:0/1723062696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0300c90c0 con 0x7ff0300c9ba0 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.481+0000 7ff02cff9640 1 -- 172.21.15.37:0/1723062696 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400fea0 con 0x7ff0300c9ba0 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.481+0000 7ff0369c9640 1 -- 172.21.15.37:0/1723062696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0300cb4d0 con 0x7ff0300c9ba0 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.482+0000 7ff0369c9640 1 -- 172.21.15.37:0/1723062696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03006a360 con 0x7ff0300c9ba0 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.482+0000 7ff02cff9640 1 -- 172.21.15.37:0/1723062696 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7ff024012040 con 0x7ff0300c9ba0 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.483+0000 7ff02cff9640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff01c06cbf0 0x7ff01c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.483+0000 7ff02cff9640 1 -- 172.21.15.37:0/1723062696 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7ff02408c360 con 0x7ff0300c9ba0 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.483+0000 7ff02ffff640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff01c06cbf0 0x7ff01c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.483+0000 7ff02ffff640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff01c06cbf0 0x7ff01c06f0b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff020033000 tx=0x7ff02002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.487+0000 7ff02cff9640 1 -- 172.21.15.37:0/1723062696 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff024019030 con 0x7ff0300c9ba0 2024-04-26T20:24:00.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.679+0000 7ff0369c9640 1 -- 172.21.15.37:0/1723062696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"} v 0) -- 0x7ff0300d2a40 con 0x7ff0300c9ba0 2024-04-26T20:24:00.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.683+0000 7ff02cff9640 1 -- 172.21.15.37:0/1723062696 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"}]=0 v12) ==== 115+0+0 (secure 0 0 0) 0x7ff02405a6b0 con 0x7ff0300c9ba0 2024-04-26T20:24:00.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 -- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff01c06cbf0 msgr2=0x7ff01c06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff01c06cbf0 0x7ff01c06f0b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff020033000 tx=0x7ff02002e000 comp rx=0 tx=0).stop 2024-04-26T20:24:00.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 -- 172.21.15.37:0/1723062696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0300c9ba0 msgr2=0x7ff0300caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0300c9ba0 0x7ff0300caa10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff024011ad0 tx=0x7ff024011f90 comp rx=0 tx=0).stop 2024-04-26T20:24:00.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 -- 172.21.15.37:0/1723062696 shutdown_connections 2024-04-26T20:24:00.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff01c06cbf0 0x7ff01c06f0b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0300caf50 0x7ff0300b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0300c9ba0 0x7ff0300caa10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 --2- 172.21.15.37:0/1723062696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0180039b0 0x7ff030156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 -- 172.21.15.37:0/1723062696 >> 172.21.15.37:0/1723062696 conn(0x7ff03009e570 msgr2=0x7ff0300b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:00.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 -- 172.21.15.37:0/1723062696 shutdown_connections 2024-04-26T20:24:00.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.685+0000 7ff0369c9640 1 -- 172.21.15.37:0/1723062696 wait complete. 2024-04-26T20:24:00.697 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T20:24:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.799+0000 7efe692ea640 1 Processor -- start 2024-04-26T20:24:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.799+0000 7efe692ea640 1 -- start start 2024-04-26T20:24:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.799+0000 7efe692ea640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efe6415f560 0x7efe640d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.799+0000 7efe692ea640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe6415f940 con 0x7efe640d10a0 2024-04-26T20:24:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.799+0000 7efe692ea640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe640d43a0 con 0x7efe640d23a0 2024-04-26T20:24:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.799+0000 7efe692ea640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7efe640d3300 con 0x7efe6415f560 2024-04-26T20:24:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe627fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7efe640d10a0 0x7efe640d1c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33230/0 (socket says 172.21.15.37:33230) 2024-04-26T20:24:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe627fc640 1 -- 172.21.15.37:0/3763802429 learned_addr learned my addr 172.21.15.37:0/3763802429 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe62ffd640 1 --2- 172.21.15.37:0/3763802429 >> v2:172.21.15.134:3300/0 conn(0x7efe6415f560 0x7efe640d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 131602441 0 0) 0x7efe6415f940 con 0x7efe640d10a0 2024-04-26T20:24:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe50003520 con 0x7efe640d10a0 2024-04-26T20:24:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 120855300 0 0) 0x7efe50003520 con 0x7efe640d10a0 2024-04-26T20:24:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 >> v2:172.21.15.134:3300/0 conn(0x7efe6415f560 msgr2=0x7efe640d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 --2- 172.21.15.37:0/3763802429 >> v2:172.21.15.134:3300/0 conn(0x7efe6415f560 0x7efe640d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 >> v1:172.21.15.134:6789/0 conn(0x7efe640d23a0 legacy=0x7efe641693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe640d35e0 con 0x7efe640d10a0 2024-04-26T20:24:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7efe4c001670 con 0x7efe640d10a0 2024-04-26T20:24:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 >> v1:172.21.15.37:6789/0 conn(0x7efe640d10a0 legacy=0x7efe640d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe500039b0 0x7efe50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 0x7efe50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe500051c0 0x7efe5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efe640d3300 con 0x7efe50004800 2024-04-26T20:24:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efe5000c060 con 0x7efe500051c0 2024-04-26T20:24:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efe5000c260 con 0x7efe500039b0 2024-04-26T20:24:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe627fc640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 0x7efe50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe62ffd640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe500039b0 0x7efe50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.800+0000 7efe637fe640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe500051c0 0x7efe5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.801+0000 7efe627fc640 1 -- 172.21.15.37:0/3763802429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe500039b0 msgr2=0x7efe50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.801+0000 7efe627fc640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe500039b0 0x7efe50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.801+0000 7efe627fc640 1 -- 172.21.15.37:0/3763802429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe500051c0 msgr2=0x7efe5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.801+0000 7efe627fc640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe500051c0 0x7efe5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.801+0000 7efe627fc640 1 -- 172.21.15.37:0/3763802429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe50003520 con 0x7efe50004800 2024-04-26T20:24:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.801+0000 7efe637fe640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe500051c0 0x7efe5000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.801+0000 7efe637fe640 1 -- 172.21.15.37:0/3763802429 reap_dead start 2024-04-26T20:24:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.801+0000 7efe627fc640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 0x7efe50004c80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c210 tx=0x7efe50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346573208f3da1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.801+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c007d30 con 0x7efe50004800 2024-04-26T20:24:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.802+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c005ad0 con 0x7efe50004800 2024-04-26T20:24:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.802+0000 7efe61ffb640 1 -- 172.21.15.37:0/3763802429 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c007070 con 0x7efe50004800 2024-04-26T20:24:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.802+0000 7efe692ea640 1 -- 172.21.15.37:0/3763802429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 msgr2=0x7efe50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.802+0000 7efe692ea640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 0x7efe50004c80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c210 tx=0x7efe50005830 comp rx=0 tx=0).stop 2024-04-26T20:24:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.802+0000 7efe692ea640 1 -- 172.21.15.37:0/3763802429 shutdown_connections 2024-04-26T20:24:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.802+0000 7efe692ea640 1 --2- 172.21.15.37:0/3763802429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 0x7efe50004c80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.802+0000 7efe692ea640 1 -- 172.21.15.37:0/3763802429 >> 172.21.15.37:0/3763802429 conn(0x7efe6409e570 msgr2=0x7efe640d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.802+0000 7efe692ea640 1 -- 172.21.15.37:0/3763802429 shutdown_connections 2024-04-26T20:24:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.802+0000 7efe692ea640 1 -- 172.21.15.37:0/3763802429 wait complete. 2024-04-26T20:24:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.803+0000 7efe692ea640 1 Processor -- start 2024-04-26T20:24:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe692ea640 1 -- start start 2024-04-26T20:24:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe692ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 0x7efe64156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe692ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe641e8e50 0x7efe640b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe692ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6415d1b0 0x7efe6415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe692ea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efe640d3e80 con 0x7efe50004800 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe692ea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efe640d3ba0 con 0x7efe641e8e50 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe692ea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efe640d2eb0 con 0x7efe6415d1b0 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe62ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 0x7efe64156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe62ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 0x7efe64156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59452/0 (socket says 172.21.15.37:59452) 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe62ffd640 1 -- 172.21.15.37:0/1147124628 learned_addr learned my addr 172.21.15.37:0/1147124628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe637fe640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6415d1b0 0x7efe6415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.804+0000 7efe627fc640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe641e8e50 0x7efe640b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe62ffd640 1 -- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6415d1b0 msgr2=0x7efe6415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe62ffd640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6415d1b0 0x7efe6415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe62ffd640 1 -- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe641e8e50 msgr2=0x7efe640b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe62ffd640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe641e8e50 0x7efe640b7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe62ffd640 1 -- 172.21.15.37:0/1147124628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe641e93a0 con 0x7efe50004800 2024-04-26T20:24:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe627fc640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe641e8e50 0x7efe640b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe62ffd640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 0x7efe64156340 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efe5c011a20 tx=0x7efe5c011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe5b7fe640 1 -- 172.21.15.37:0/1147124628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c00fbb0 con 0x7efe50004800 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe5b7fe640 1 -- 172.21.15.37:0/1147124628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5c00fd50 con 0x7efe50004800 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe692ea640 1 -- 172.21.15.37:0/1147124628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe641e9680 con 0x7efe50004800 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe5b7fe640 1 -- 172.21.15.37:0/1147124628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c014640 con 0x7efe50004800 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.805+0000 7efe692ea640 1 -- 172.21.15.37:0/1147124628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6415fb30 con 0x7efe50004800 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.806+0000 7efe692ea640 1 -- 172.21.15.37:0/1147124628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6415db70 con 0x7efe50004800 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.807+0000 7efe5b7fe640 1 -- 172.21.15.37:0/1147124628 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7efe5c012040 con 0x7efe50004800 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.807+0000 7efe5b7fe640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe4406cbf0 0x7efe4406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.807+0000 7efe5b7fe640 1 -- 172.21.15.37:0/1147124628 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7efe5c08c3f0 con 0x7efe50004800 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.807+0000 7efe627fc640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe4406cbf0 0x7efe4406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.808+0000 7efe627fc640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe4406cbf0 0x7efe4406f0b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efe4c033000 tx=0x7efe4c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.811+0000 7efe5b7fe640 1 -- 172.21.15.37:0/1147124628 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe5c019030 con 0x7efe50004800 2024-04-26T20:24:00.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.996+0000 7efe692ea640 1 -- 172.21.15.37:0/1147124628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7efe6406a360 con 0x7efe50004800 2024-04-26T20:24:00.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:00.999+0000 7efe5b7fe640 1 -- 172.21.15.37:0/1147124628 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7efe5c05a810 con 0x7efe50004800 2024-04-26T20:24:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 -- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe4406cbf0 msgr2=0x7efe4406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe4406cbf0 0x7efe4406f0b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efe4c033000 tx=0x7efe4c02e000 comp rx=0 tx=0).stop 2024-04-26T20:24:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 -- 172.21.15.37:0/1147124628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 msgr2=0x7efe64156340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:01.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 0x7efe64156340 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efe5c011a20 tx=0x7efe5c011ee0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 -- 172.21.15.37:0/1147124628 shutdown_connections 2024-04-26T20:24:01.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7efe4406cbf0 0x7efe4406f0b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6415d1b0 0x7efe6415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe641e8e50 0x7efe640b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 --2- 172.21.15.37:0/1147124628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe50004800 0x7efe64156340 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 -- 172.21.15.37:0/1147124628 >> 172.21.15.37:0/1147124628 conn(0x7efe6409e570 msgr2=0x7efe640d1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:01.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 -- 172.21.15.37:0/1147124628 shutdown_connections 2024-04-26T20:24:01.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.001+0000 7efe692ea640 1 -- 172.21.15.37:0/1147124628 wait complete. 2024-04-26T20:24:01.013 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed' 2024-04-26T20:24:01.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.113+0000 7f4c0d59d640 1 Processor -- start 2024-04-26T20:24:01.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.113+0000 7f4c0d59d640 1 -- start start 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.114+0000 7f4c0d59d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c0806ea40 0x7f4c080d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.114+0000 7f4c0d59d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c080b1d70 con 0x7f4c08162e40 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.114+0000 7f4c0d59d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c080d1960 con 0x7f4c080cf830 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.114+0000 7f4c0d59d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4c08163220 con 0x7f4c0806ea40 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.114+0000 7f4c06ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4c08162e40 0x7f4c080b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33242/0 (socket says 172.21.15.37:33242) 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.114+0000 7f4c06ffd640 1 -- 172.21.15.37:0/1497329225 learned_addr learned my addr 172.21.15.37:0/1497329225 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.114+0000 7f4c067fc640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c0806ea40 0x7f4c080d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.114+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2632240795 0 0) 0x7f4c080b1d70 con 0x7f4c08162e40 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.114+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bf0003520 con 0x7f4c08162e40 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 241960294 0 0) 0x7f4bf0003520 con 0x7f4c08162e40 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c0806ea40 msgr2=0x7f4c080d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:01.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c0806ea40 0x7f4c080d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 >> v1:172.21.15.134:6789/0 conn(0x7f4c080cf830 legacy=0x7f4c080ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c080d1c40 con 0x7f4c08162e40 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f4bf8001670 con 0x7f4c08162e40 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 >> v1:172.21.15.37:6789/0 conn(0x7f4c08162e40 legacy=0x7f4c080b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf00039e0 0x7f4bf0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c0806ea40 0x7f4bf0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf000baf0 con 0x7f4bf00039e0 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf000bca0 con 0x7f4bf0004800 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4bf000bea0 con 0x7f4c0806ea40 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c06ffd640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf00039e0 0x7f4bf0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c067fc640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c077fe640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c0806ea40 0x7f4bf0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c06ffd640 1 -- 172.21.15.37:0/1497329225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c0806ea40 msgr2=0x7f4bf0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c06ffd640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c0806ea40 0x7f4bf0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c06ffd640 1 -- 172.21.15.37:0/1497329225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf0004800 msgr2=0x7f4bf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c06ffd640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.115+0000 7f4c06ffd640 1 -- 172.21.15.37:0/1497329225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf0003520 con 0x7f4bf00039e0 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.116+0000 7f4c06ffd640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf00039e0 0x7f4bf0003e00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800c210 tx=0x7f4bf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6506e89dbd35a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.116+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8007d30 con 0x7f4bf00039e0 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.116+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf8005ad0 con 0x7f4bf00039e0 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.116+0000 7f4c05ffb640 1 -- 172.21.15.37:0/1497329225 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8007070 con 0x7f4bf00039e0 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.116+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1497329225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf00039e0 msgr2=0x7f4bf0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.116+0000 7f4c0d59d640 1 --2- 172.21.15.37:0/1497329225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf00039e0 0x7f4bf0003e00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800c210 tx=0x7f4bf0005830 comp rx=0 tx=0).stop 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.116+0000 7f4c077fe640 1 -- 172.21.15.37:0/1497329225 reap_dead start 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.116+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1497329225 shutdown_connections 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.116+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1497329225 >> 172.21.15.37:0/1497329225 conn(0x7f4c0809ba50 msgr2=0x7f4c0809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.117+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1497329225 shutdown_connections 2024-04-26T20:24:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.117+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1497329225 wait complete. 2024-04-26T20:24:01.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.118+0000 7f4c0d59d640 1 Processor -- start 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.118+0000 7f4c0d59d640 1 -- start start 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c0d59d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bf00039e0 0x7f4c08158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c0d59d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c080b3fa0 0x7f4c080b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c0d59d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0815e2a0 0x7f4c0815e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c0d59d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4c080d1340 con 0x7f4c0815e2a0 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c0d59d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4c080d0eb0 con 0x7f4c080b3fa0 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c0d59d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4c080d14c0 con 0x7f4bf00039e0 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0815e2a0 0x7f4c0815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0815e2a0 0x7f4c0815e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59458/0 (socket says 172.21.15.37:59458) 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c077fe640 1 -- 172.21.15.37:0/1573158644 learned_addr learned my addr 172.21.15.37:0/1573158644 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c06ffd640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bf00039e0 0x7f4c08158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c067fc640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c080b3fa0 0x7f4c080b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c077fe640 1 -- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bf00039e0 msgr2=0x7f4c08158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:01.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c077fe640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bf00039e0 0x7f4c08158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c077fe640 1 -- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c080b3fa0 msgr2=0x7f4c080b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c077fe640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c080b3fa0 0x7f4c080b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c077fe640 1 -- 172.21.15.37:0/1573158644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0815a9e0 con 0x7f4c0815e2a0 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c067fc640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c080b3fa0 0x7f4c080b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c077fe640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0815e2a0 0x7f4c0815e720 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4c0809c990 tx=0x7f4c0000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4bef7fe640 1 -- 172.21.15.37:0/1573158644 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00003970 con 0x7f4c0815e2a0 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4bef7fe640 1 -- 172.21.15.37:0/1573158644 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c00018070 con 0x7f4c0815e2a0 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1573158644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0815acc0 con 0x7f4c0815e2a0 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4bef7fe640 1 -- 172.21.15.37:0/1573158644 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00003d50 con 0x7f4c0815e2a0 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.119+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1573158644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0815b280 con 0x7f4c0815e2a0 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.120+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1573158644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c0806a360 con 0x7f4c0815e2a0 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.121+0000 7f4bef7fe640 1 -- 172.21.15.37:0/1573158644 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f4c000050d0 con 0x7f4c0815e2a0 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.121+0000 7f4bef7fe640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bf406cbc0 0x7f4bf406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.121+0000 7f4bef7fe640 1 -- 172.21.15.37:0/1573158644 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f4c0008b890 con 0x7f4c0815e2a0 2024-04-26T20:24:01.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.121+0000 7f4c06ffd640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bf406cbc0 0x7f4bf406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:01.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.122+0000 7f4c06ffd640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bf406cbc0 0x7f4bf406f080 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8033000 tx=0x7f4bf802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:01.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.125+0000 7f4bef7fe640 1 -- 172.21.15.37:0/1573158644 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c00059d00 con 0x7f4c0815e2a0 2024-04-26T20:24:01.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:01.385+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1573158644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed"]} v 0) -- 0x7f4c08158a10 con 0x7f4c0815e2a0 2024-04-26T20:24:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.353+0000 7f4bef7fe640 1 -- 172.21.15.37:0/1573158644 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed"]}]=0 v135) ==== 165+0+0 (secure 0 0 0) 0x7f4c00059600 con 0x7f4c0815e2a0 2024-04-26T20:24:02.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.355+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bf406cbc0 msgr2=0x7f4bf406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:02.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.355+0000 7f4c0d59d640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bf406cbc0 0x7f4bf406f080 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8033000 tx=0x7f4bf802e000 comp rx=0 tx=0).stop 2024-04-26T20:24:02.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.355+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1573158644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0815e2a0 msgr2=0x7f4c0815e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:02.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.355+0000 7f4c0d59d640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0815e2a0 0x7f4c0815e720 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4c0809c990 tx=0x7f4c0000ec10 comp rx=0 tx=0).stop 2024-04-26T20:24:02.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.355+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1573158644 shutdown_connections 2024-04-26T20:24:02.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.355+0000 7f4c0d59d640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0815e2a0 0x7f4c0815e720 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.355+0000 7f4c0d59d640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c080b3fa0 0x7f4c080b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.355+0000 7f4c0d59d640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4bf406cbc0 0x7f4bf406f080 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.355+0000 7f4c0d59d640 1 --2- 172.21.15.37:0/1573158644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bf00039e0 0x7f4c08158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.355+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1573158644 >> 172.21.15.37:0/1573158644 conn(0x7f4c0809ba50 msgr2=0x7f4c0815f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:02.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.356+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1573158644 shutdown_connections 2024-04-26T20:24:02.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.356+0000 7f4c0d59d640 1 -- 172.21.15.37:0/1573158644 wait complete. 2024-04-26T20:24:02.370 INFO:tasks.cephfs_test_runner:ensure failure status is shown when clone is in failed state and validate the reason ... ok 2024-04-26T20:24:02.371 INFO:tasks.cephfs_test_runner:Starting test: test_clone_failure_status_in_progress_cancelled (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) ensure failure status is shown when clone is cancelled during in-progress state and validate the reason 2024-04-26T20:24:02.371 INFO:tasks.cephfs_test_runner:test_clone_failure_status_in_progress_cancelled (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2024-04-26T20:24:02.371 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:24:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.477+0000 7f19a11bd640 1 Processor -- start 2024-04-26T20:24:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.477+0000 7f19a11bd640 1 -- start start 2024-04-26T20:24:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f19a11bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f199c06ea40 0x7f199c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f19a11bd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f199c0cf830 0x7f199c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f19a11bd640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f199c0b1d70 con 0x7f199c162e40 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f19a11bd640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f199c0d01e0 con 0x7f199c0cf830 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f19a11bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f199c163220 con 0x7f199c06ea40 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f199b59e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f199c0cf830 0x7f199c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f199ad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f199c162e40 0x7f199c0b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33246/0 (socket says 172.21.15.37:33246) 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f199ad9d640 1 -- 172.21.15.37:0/1444613708 learned_addr learned my addr 172.21.15.37:0/1444613708 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f1993fff640 1 --2- 172.21.15.37:0/1444613708 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f199c06ea40 0x7f199c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f199b59e640 1 -- 172.21.15.37:0/1444613708 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f199c06ea40 msgr2=0x7f199c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f199b59e640 1 --2- 172.21.15.37:0/1444613708 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f199c06ea40 0x7f199c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f199b59e640 1 -- 172.21.15.37:0/1444613708 >> v1:172.21.15.37:6789/0 conn(0x7f199c162e40 legacy=0x7f199c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:02.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f199a59c640 1 -- 172.21.15.37:0/1444613708 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 225621435 0 0) 0x7f199c0b1d70 con 0x7f199c162e40 2024-04-26T20:24:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f199b59e640 1 -- 172.21.15.37:0/1444613708 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c0d1960 con 0x7f199c0cf830 2024-04-26T20:24:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.478+0000 7f1993fff640 1 --2- 172.21.15.37:0/1444613708 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f199c06ea40 0x7f199c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.479+0000 7f199b59e640 1 --2- 172.21.15.37:0/1444613708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f199c0cf830 0x7f199c0cfca0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1994009d90 tx=0x7f199402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50ce633ef8af487 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.479+0000 7f199a59c640 1 -- 172.21.15.37:0/1444613708 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994031600 con 0x7f199c0cf830 2024-04-26T20:24:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.479+0000 7f199a59c640 1 -- 172.21.15.37:0/1444613708 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19940317a0 con 0x7f199c0cf830 2024-04-26T20:24:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.479+0000 7f199a59c640 1 -- 172.21.15.37:0/1444613708 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994031b70 con 0x7f199c0cf830 2024-04-26T20:24:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.480+0000 7f19a11bd640 1 -- 172.21.15.37:0/1444613708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f199c0cf830 msgr2=0x7f199c0cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.480+0000 7f19a11bd640 1 --2- 172.21.15.37:0/1444613708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f199c0cf830 0x7f199c0cfca0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1994009d90 tx=0x7f199402a5d0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.480+0000 7f19a11bd640 1 -- 172.21.15.37:0/1444613708 shutdown_connections 2024-04-26T20:24:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.480+0000 7f19a11bd640 1 --2- 172.21.15.37:0/1444613708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f199c0cf830 0x7f199c0cfca0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.480+0000 7f19a11bd640 1 --2- 172.21.15.37:0/1444613708 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f199c06ea40 0x7f199c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.480+0000 7f19a11bd640 1 -- 172.21.15.37:0/1444613708 >> 172.21.15.37:0/1444613708 conn(0x7f199c09ba50 msgr2=0x7f199c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.480+0000 7f19a11bd640 1 -- 172.21.15.37:0/1444613708 shutdown_connections 2024-04-26T20:24:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.480+0000 7f19a11bd640 1 -- 172.21.15.37:0/1444613708 wait complete. 2024-04-26T20:24:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.481+0000 7f19a11bd640 1 Processor -- start 2024-04-26T20:24:02.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.481+0000 7f19a11bd640 1 -- start start 2024-04-26T20:24:02.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f19a11bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f199c06ea40 0x7f199c0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f19a11bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f199c0cf830 0x7f199c1f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f19a11bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f199c162e40 0x7f199c1f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f19a11bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f199c0d1f20 con 0x7f199c162e40 2024-04-26T20:24:02.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f19a11bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f199c0d0eb0 con 0x7f199c06ea40 2024-04-26T20:24:02.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f19a11bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f199c0ce010 con 0x7f199c0cf830 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f199b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f199c162e40 0x7f199c1f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f199b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f199c162e40 0x7f199c1f8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59480/0 (socket says 172.21.15.37:59480) 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f199b59e640 1 -- 172.21.15.37:0/983187737 learned_addr learned my addr 172.21.15.37:0/983187737 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f1993fff640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f199c0cf830 0x7f199c1f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f199b59e640 1 -- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f199c0cf830 msgr2=0x7f199c1f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f199ad9d640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f199c06ea40 0x7f199c0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f199b59e640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f199c0cf830 0x7f199c1f8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f199b59e640 1 -- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f199c06ea40 msgr2=0x7f199c0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f199b59e640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f199c06ea40 0x7f199c0b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.482+0000 7f199b59e640 1 -- 172.21.15.37:0/983187737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c159a40 con 0x7f199c162e40 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.483+0000 7f199b59e640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f199c162e40 0x7f199c1f8e80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1994009eb0 tx=0x7f1994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.483+0000 7f19937fe640 1 -- 172.21.15.37:0/983187737 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994040070 con 0x7f199c162e40 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.483+0000 7f19937fe640 1 -- 172.21.15.37:0/983187737 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1994030b60 con 0x7f199c162e40 2024-04-26T20:24:02.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.483+0000 7f19a11bd640 1 -- 172.21.15.37:0/983187737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199c0b0600 con 0x7f199c162e40 2024-04-26T20:24:02.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.483+0000 7f19937fe640 1 -- 172.21.15.37:0/983187737 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994045460 con 0x7f199c162e40 2024-04-26T20:24:02.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.483+0000 7f19a11bd640 1 -- 172.21.15.37:0/983187737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199c0b0bc0 con 0x7f199c162e40 2024-04-26T20:24:02.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.484+0000 7f19a11bd640 1 -- 172.21.15.37:0/983187737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f199c06a360 con 0x7f199c162e40 2024-04-26T20:24:02.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.484+0000 7f19937fe640 1 -- 172.21.15.37:0/983187737 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f1994039040 con 0x7f199c162e40 2024-04-26T20:24:02.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.485+0000 7f19937fe640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f197c06cbc0 0x7f197c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.485+0000 7f19937fe640 1 -- 172.21.15.37:0/983187737 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f19940ad770 con 0x7f199c162e40 2024-04-26T20:24:02.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.485+0000 7f199ad9d640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f197c06cbc0 0x7f197c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.486+0000 7f199ad9d640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f197c06cbc0 0x7f197c06f080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f198c000f60 tx=0x7f198c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:02.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.489+0000 7f19937fe640 1 -- 172.21.15.37:0/983187737 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f199407b9e0 con 0x7f199c162e40 2024-04-26T20:24:02.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.679+0000 7f19a11bd640 1 -- 172.21.15.37:0/983187737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f199c06f030 con 0x7f199c162e40 2024-04-26T20:24:02.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.680+0000 7f19937fe640 1 -- 172.21.15.37:0/983187737 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13028 (secure 0 0 0) 0x7f199407b2e0 con 0x7f199c162e40 2024-04-26T20:24:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:02.681 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":38,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:24:00.343481+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6834/64661056":"2024-04-27T20:23:57.334082+0000","172.21.15.37:6832/64661056":"2024-04-27T20:23:57.334082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:24:02.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 -- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f197c06cbc0 msgr2=0x7f197c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f197c06cbc0 0x7f197c06f080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f198c000f60 tx=0x7f198c009040 comp rx=0 tx=0).stop 2024-04-26T20:24:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 -- 172.21.15.37:0/983187737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f199c162e40 msgr2=0x7f199c1f8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f199c162e40 0x7f199c1f8e80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1994009eb0 tx=0x7f1994004a40 comp rx=0 tx=0).stop 2024-04-26T20:24:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 -- 172.21.15.37:0/983187737 shutdown_connections 2024-04-26T20:24:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f199c162e40 0x7f199c1f8e80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f197c06cbc0 0x7f197c06f080 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f199c0cf830 0x7f199c1f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 --2- 172.21.15.37:0/983187737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f199c06ea40 0x7f199c0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 -- 172.21.15.37:0/983187737 >> 172.21.15.37:0/983187737 conn(0x7f199c09ba50 msgr2=0x7f199c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 -- 172.21.15.37:0/983187737 shutdown_connections 2024-04-26T20:24:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.682+0000 7f19a11bd640 1 -- 172.21.15.37:0/983187737 wait complete. 2024-04-26T20:24:02.695 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:24:02.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:24:02.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.798+0000 7f83fd9e7640 1 Processor -- start 2024-04-26T20:24:02.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.798+0000 7f83fd9e7640 1 -- start start 2024-04-26T20:24:02.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83fd9e7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83f815f470 0x7f83f80d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83fd9e7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f83f80d0d30 0x7f83f80d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83fd9e7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83f80d2ed0 con 0x7f83f80d1fe0 2024-04-26T20:24:02.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83fd9e7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f83f80d1190 con 0x7f83f815f470 2024-04-26T20:24:02.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83fd9e7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f83f80d31b0 con 0x7f83f80d0d30 2024-04-26T20:24:02.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83f6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83f815f470 0x7f83f80d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83f6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83f815f470 0x7f83f80d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59482/0 (socket says 172.21.15.37:59482) 2024-04-26T20:24:02.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83f6ffd640 1 -- 172.21.15.37:0/2729585336 learned_addr learned my addr 172.21.15.37:0/2729585336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:02.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83f67fc640 1 --2- 172.21.15.37:0/2729585336 >> v2:172.21.15.134:3300/0 conn(0x7f83f80d0d30 0x7f83f80d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83f6ffd640 1 -- 172.21.15.37:0/2729585336 >> v2:172.21.15.134:3300/0 conn(0x7f83f80d0d30 msgr2=0x7f83f80d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:02.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83f6ffd640 1 --2- 172.21.15.37:0/2729585336 >> v2:172.21.15.134:3300/0 conn(0x7f83f80d0d30 0x7f83f80d1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83f6ffd640 1 -- 172.21.15.37:0/2729585336 >> v1:172.21.15.134:6789/0 conn(0x7f83f80d1fe0 legacy=0x7f83f8162fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:02.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.799+0000 7f83f6ffd640 1 -- 172.21.15.37:0/2729585336 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f80d3490 con 0x7f83f815f470 2024-04-26T20:24:02.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.800+0000 7f83f6ffd640 1 --2- 172.21.15.37:0/2729585336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f815f470 0x7f83f80d07f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f83f80bb210 tx=0x7f83e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0fa302dcf16f5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:02.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.800+0000 7f83f5ffb640 1 -- 172.21.15.37:0/2729585336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8037070 con 0x7f83f815f470 2024-04-26T20:24:02.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.800+0000 7f83f5ffb640 1 -- 172.21.15.37:0/2729585336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e8032440 con 0x7f83f815f470 2024-04-26T20:24:02.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.800+0000 7f83f5ffb640 1 -- 172.21.15.37:0/2729585336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8032810 con 0x7f83f815f470 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.800+0000 7f83fd9e7640 1 -- 172.21.15.37:0/2729585336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f815f470 msgr2=0x7f83f80d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.800+0000 7f83fd9e7640 1 --2- 172.21.15.37:0/2729585336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f815f470 0x7f83f80d07f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f83f80bb210 tx=0x7f83e802a230 comp rx=0 tx=0).stop 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.800+0000 7f83fd9e7640 1 -- 172.21.15.37:0/2729585336 shutdown_connections 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.800+0000 7f83fd9e7640 1 --2- 172.21.15.37:0/2729585336 >> v2:172.21.15.134:3300/0 conn(0x7f83f80d0d30 0x7f83f80d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.800+0000 7f83fd9e7640 1 --2- 172.21.15.37:0/2729585336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f815f470 0x7f83f80d07f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.800+0000 7f83fd9e7640 1 -- 172.21.15.37:0/2729585336 >> 172.21.15.37:0/2729585336 conn(0x7f83f809e510 msgr2=0x7f83f80d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.801+0000 7f83fd9e7640 1 -- 172.21.15.37:0/2729585336 shutdown_connections 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.801+0000 7f83fd9e7640 1 -- 172.21.15.37:0/2729585336 wait complete. 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.802+0000 7f83fd9e7640 1 Processor -- start 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.802+0000 7f83fd9e7640 1 -- start start 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83fd9e7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f80d0d30 0x7f83f8153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83fd9e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f80d1fe0 0x7f83f80cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83fd9e7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83f815f470 0x7f83f80cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83fd9e7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f83f815f850 con 0x7f83f80d0d30 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83fd9e7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83f80d3a50 con 0x7f83f80d1fe0 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83fd9e7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f83f80d3f70 con 0x7f83f815f470 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f80d0d30 0x7f83f8153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f80d0d30 0x7f83f8153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59492/0 (socket says 172.21.15.37:59492) 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f6ffd640 1 -- 172.21.15.37:0/525482249 learned_addr learned my addr 172.21.15.37:0/525482249 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f77fe640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83f815f470 0x7f83f80cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f67fc640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f80d1fe0 0x7f83f80cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f6ffd640 1 -- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83f815f470 msgr2=0x7f83f80cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f6ffd640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83f815f470 0x7f83f80cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f6ffd640 1 -- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f80d1fe0 msgr2=0x7f83f80cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f6ffd640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f80d1fe0 0x7f83f80cad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f6ffd640 1 -- 172.21.15.37:0/525482249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f80cd240 con 0x7f83f80d0d30 2024-04-26T20:24:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f77fe640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83f815f470 0x7f83f80cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83f6ffd640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f80d0d30 0x7f83f8153a00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f83e8000c00 tx=0x7f83e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83dffff640 1 -- 172.21.15.37:0/525482249 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8041070 con 0x7f83f80d0d30 2024-04-26T20:24:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83dffff640 1 -- 172.21.15.37:0/525482249 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e803c870 con 0x7f83f80d0d30 2024-04-26T20:24:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83fd9e7640 1 -- 172.21.15.37:0/525482249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f80cc9a0 con 0x7f83f80d0d30 2024-04-26T20:24:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83dffff640 1 -- 172.21.15.37:0/525482249 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8037040 con 0x7f83f80d0d30 2024-04-26T20:24:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.803+0000 7f83fd9e7640 1 -- 172.21.15.37:0/525482249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f80ccf60 con 0x7f83f80d0d30 2024-04-26T20:24:02.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.804+0000 7f83fd9e7640 1 -- 172.21.15.37:0/525482249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f806a360 con 0x7f83f80d0d30 2024-04-26T20:24:02.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.805+0000 7f83dffff640 1 -- 172.21.15.37:0/525482249 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f83e8044070 con 0x7f83f80d0d30 2024-04-26T20:24:02.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.805+0000 7f83dffff640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83e406cbc0 0x7f83e406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:02.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.806+0000 7f83dffff640 1 -- 172.21.15.37:0/525482249 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f83e80ac960 con 0x7f83f80d0d30 2024-04-26T20:24:02.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.806+0000 7f83f67fc640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83e406cbc0 0x7f83e406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.806+0000 7f83f67fc640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83e406cbc0 0x7f83e406f080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f83f80cb620 tx=0x7f83ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:02.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.810+0000 7f83dffff640 1 -- 172.21.15.37:0/525482249 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f83e807ae10 con 0x7f83f80d0d30 2024-04-26T20:24:02.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.996+0000 7f83fd9e7640 1 -- 172.21.15.37:0/525482249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83f80d5870 con 0x7f83f80d0d30 2024-04-26T20:24:02.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.997+0000 7f83dffff640 1 -- 172.21.15.37:0/525482249 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f83e807a710 con 0x7f83f80d0d30 2024-04-26T20:24:02.998 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:24:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 -- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83e406cbc0 msgr2=0x7f83e406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83e406cbc0 0x7f83e406f080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f83f80cb620 tx=0x7f83ec008040 comp rx=0 tx=0).stop 2024-04-26T20:24:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 -- 172.21.15.37:0/525482249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f80d0d30 msgr2=0x7f83f8153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f80d0d30 0x7f83f8153a00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f83e8000c00 tx=0x7f83e80046c0 comp rx=0 tx=0).stop 2024-04-26T20:24:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 -- 172.21.15.37:0/525482249 shutdown_connections 2024-04-26T20:24:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f83e406cbc0 0x7f83e406f080 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83f815f470 0x7f83f80cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f80d1fe0 0x7f83f80cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 --2- 172.21.15.37:0/525482249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f80d0d30 0x7f83f8153a00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 -- 172.21.15.37:0/525482249 >> 172.21.15.37:0/525482249 conn(0x7f83f809e510 msgr2=0x7f83f815d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 -- 172.21.15.37:0/525482249 shutdown_connections 2024-04-26T20:24:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:02.999+0000 7f83fd9e7640 1 -- 172.21.15.37:0/525482249 wait complete. 2024-04-26T20:24:03.011 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_in_progress_cancelled' 2024-04-26T20:24:03.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.110+0000 7f4958578640 1 Processor -- start 2024-04-26T20:24:03.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.110+0000 7f4958578640 1 -- start start 2024-04-26T20:24:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.111+0000 7f4958578640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4950159a90 0x7f495006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.111+0000 7f4958578640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4950160320 0x7f49500ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.111+0000 7f4958578640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4950072e30 con 0x7f4950070130 2024-04-26T20:24:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.111+0000 7f4958578640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4950073110 con 0x7f4950160320 2024-04-26T20:24:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.111+0000 7f4958578640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f49500733f0 con 0x7f4950159a90 2024-04-26T20:24:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.112+0000 7f4956b15640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4950160320 0x7f49500ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.112+0000 7f4956b15640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4950160320 0x7f49500ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59502/0 (socket says 172.21.15.37:59502) 2024-04-26T20:24:03.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.112+0000 7f4956b15640 1 -- 172.21.15.37:0/4058849307 learned_addr learned my addr 172.21.15.37:0/4058849307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:03.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.112+0000 7f4956314640 1 --2- 172.21.15.37:0/4058849307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4950159a90 0x7f495006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:03.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.112+0000 7f4956b15640 1 -- 172.21.15.37:0/4058849307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4950159a90 msgr2=0x7f495006fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:03.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.112+0000 7f4956b15640 1 --2- 172.21.15.37:0/4058849307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4950159a90 0x7f495006fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:03.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.112+0000 7f4956b15640 1 -- 172.21.15.37:0/4058849307 >> v1:172.21.15.134:6789/0 conn(0x7f4950070130 legacy=0x7f495015fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:03.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.112+0000 7f4956b15640 1 -- 172.21.15.37:0/4058849307 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49500736d0 con 0x7f4950160320 2024-04-26T20:24:03.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4956b15640 1 --2- 172.21.15.37:0/4058849307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4950160320 0x7f49500ce9b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f494800bb10 tx=0x7f494802c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933c1f37535ccf1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:03.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4955312640 1 -- 172.21.15.37:0/4058849307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948031710 con 0x7f4950160320 2024-04-26T20:24:03.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4955312640 1 -- 172.21.15.37:0/4058849307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49480318b0 con 0x7f4950160320 2024-04-26T20:24:03.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4955312640 1 -- 172.21.15.37:0/4058849307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948031c80 con 0x7f4950160320 2024-04-26T20:24:03.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4958578640 1 -- 172.21.15.37:0/4058849307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4950160320 msgr2=0x7f49500ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:03.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4958578640 1 --2- 172.21.15.37:0/4058849307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4950160320 0x7f49500ce9b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f494800bb10 tx=0x7f494802c7a0 comp rx=0 tx=0).stop 2024-04-26T20:24:03.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4958578640 1 -- 172.21.15.37:0/4058849307 shutdown_connections 2024-04-26T20:24:03.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4958578640 1 --2- 172.21.15.37:0/4058849307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4950160320 0x7f49500ce9b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:03.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4958578640 1 --2- 172.21.15.37:0/4058849307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4950159a90 0x7f495006fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:03.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4958578640 1 -- 172.21.15.37:0/4058849307 >> 172.21.15.37:0/4058849307 conn(0x7f49500d0d50 msgr2=0x7f49500d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:03.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4958578640 1 -- 172.21.15.37:0/4058849307 shutdown_connections 2024-04-26T20:24:03.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.113+0000 7f4958578640 1 -- 172.21.15.37:0/4058849307 wait complete. 2024-04-26T20:24:03.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.115+0000 7f4958578640 1 Processor -- start 2024-04-26T20:24:03.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.115+0000 7f4958578640 1 -- start start 2024-04-26T20:24:03.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.115+0000 7f4958578640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4950070130 0x7f49500b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:03.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.115+0000 7f4958578640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4950159a90 0x7f495015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:03.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.115+0000 7f4958578640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4950160320 0x7f495013f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:03.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.115+0000 7f4958578640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4950159ff0 con 0x7f4950159a90 2024-04-26T20:24:03.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.115+0000 7f4958578640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4950159e70 con 0x7f4950160320 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.115+0000 7f4958578640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f49500ceef0 con 0x7f4950070130 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.115+0000 7f4955b13640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4950159a90 0x7f495015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.115+0000 7f4955b13640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4950159a90 0x7f495015cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59504/0 (socket says 172.21.15.37:59504) 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f4955b13640 1 -- 172.21.15.37:0/2721156375 learned_addr learned my addr 172.21.15.37:0/2721156375 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f4956314640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4950070130 0x7f49500b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f4956b15640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4950160320 0x7f495013f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f4955b13640 1 -- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4950070130 msgr2=0x7f49500b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f4955b13640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4950070130 0x7f49500b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f4955b13640 1 -- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4950160320 msgr2=0x7f495013f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f4955b13640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4950160320 0x7f495013f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f4955b13640 1 -- 172.21.15.37:0/2721156375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49500b5200 con 0x7f4950159a90 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f4955b13640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4950159a90 0x7f495015cbc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f494400c460 tx=0x7f494400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f494effd640 1 -- 172.21.15.37:0/2721156375 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944016020 con 0x7f4950159a90 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f494effd640 1 -- 172.21.15.37:0/2721156375 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4944012870 con 0x7f4950159a90 2024-04-26T20:24:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f494effd640 1 -- 172.21.15.37:0/2721156375 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944012c60 con 0x7f4950159a90 2024-04-26T20:24:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f4958578640 1 -- 172.21.15.37:0/2721156375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49500b54e0 con 0x7f4950159a90 2024-04-26T20:24:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.116+0000 7f4958578640 1 -- 172.21.15.37:0/2721156375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49500b5aa0 con 0x7f4950159a90 2024-04-26T20:24:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.117+0000 7f4958578640 1 -- 172.21.15.37:0/2721156375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495006a360 con 0x7f4950159a90 2024-04-26T20:24:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.118+0000 7f494effd640 1 -- 172.21.15.37:0/2721156375 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f4944019020 con 0x7f4950159a90 2024-04-26T20:24:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.118+0000 7f494effd640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f493c06cbc0 0x7f493c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.118+0000 7f494effd640 1 -- 172.21.15.37:0/2721156375 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f494408b940 con 0x7f4950159a90 2024-04-26T20:24:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.118+0000 7f4956314640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f493c06cbc0 0x7f493c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.119+0000 7f4956314640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f493c06cbc0 0x7f493c06f080 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4940003b80 tx=0x7f49400098d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.122+0000 7f494effd640 1 -- 172.21.15.37:0/2721156375 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4944020030 con 0x7f4950159a90 2024-04-26T20:24:03.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:03.379+0000 7f4958578640 1 -- 172.21.15.37:0/2721156375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_in_progress_cancelled"]} v 0) -- 0x7f49500d0730 con 0x7f4950159a90 2024-04-26T20:24:04.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.361+0000 7f494effd640 1 -- 172.21.15.37:0/2721156375 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_in_progress_cancelled"]}]=0 v137) ==== 183+0+0 (secure 0 0 0) 0x7f4944059640 con 0x7f4950159a90 2024-04-26T20:24:04.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 -- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f493c06cbc0 msgr2=0x7f493c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:04.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f493c06cbc0 0x7f493c06f080 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4940003b80 tx=0x7f49400098d0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 -- 172.21.15.37:0/2721156375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4950159a90 msgr2=0x7f495015cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:04.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4950159a90 0x7f495015cbc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f494400c460 tx=0x7f494400c920 comp rx=0 tx=0).stop 2024-04-26T20:24:04.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 -- 172.21.15.37:0/2721156375 shutdown_connections 2024-04-26T20:24:04.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4950160320 0x7f495013f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4950159a90 0x7f495015cbc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f493c06cbc0 0x7f493c06f080 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 --2- 172.21.15.37:0/2721156375 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4950070130 0x7f49500b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 -- 172.21.15.37:0/2721156375 >> 172.21.15.37:0/2721156375 conn(0x7f49500d0d50 msgr2=0x7f4950157ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:04.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 -- 172.21.15.37:0/2721156375 shutdown_connections 2024-04-26T20:24:04.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.364+0000 7f4958578640 1 -- 172.21.15.37:0/2721156375 wait complete. 2024-04-26T20:24:04.379 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:24:04.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.488+0000 7fc6be739640 1 Processor -- start 2024-04-26T20:24:04.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.488+0000 7fc6be739640 1 -- start start 2024-04-26T20:24:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.489+0000 7fc6be739640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b815f9c0 0x7fc6b80d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.489+0000 7fc6be739640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6b80b2550 con 0x7fc6b80cfdd0 2024-04-26T20:24:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.489+0000 7fc6be739640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6b81641a0 con 0x7fc6b80cf990 2024-04-26T20:24:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.489+0000 7fc6be739640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b80b3760 con 0x7fc6b815f9c0 2024-04-26T20:24:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.489+0000 7fc6bccd6640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc6b80cfdd0 0x7fc6b8163a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33258/0 (socket says 172.21.15.37:33258) 2024-04-26T20:24:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.489+0000 7fc6bccd6640 1 -- 172.21.15.37:0/3878709247 learned_addr learned my addr 172.21.15.37:0/3878709247 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.489+0000 7fc6b7fff640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b815f9c0 0x7fc6b80d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.489+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3541577589 0 0) 0x7fc6b0001490 con 0x7fc6b80cfdd0 2024-04-26T20:24:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.489+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6a0003520 con 0x7fc6b80cfdd0 2024-04-26T20:24:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.489+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4063630452 0 0) 0x7fc6a0003520 con 0x7fc6b80cfdd0 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b815f9c0 msgr2=0x7fc6b80d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b815f9c0 0x7fc6b80d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 >> v1:172.21.15.134:6789/0 conn(0x7fc6b80cf990 legacy=0x7fc6b8163380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b806dcf0 con 0x7fc6b80cfdd0 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fc6b0001bc0 con 0x7fc6b80cfdd0 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 >> v1:172.21.15.37:6789/0 conn(0x7fc6b80cfdd0 legacy=0x7fc6b8163a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b815f9c0 0x7fc6a00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a0003f60 0x7fc6a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6a0004d40 0x7fc6a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b80b3760 con 0x7fc6a0004d40 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a000bb50 con 0x7fc6a0003f60 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc6a000bd50 con 0x7fc6b815f9c0 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6bccd6640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6a0004d40 0x7fc6a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b7fff640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b815f9c0 0x7fc6a00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:04.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6b77fe640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a0003f60 0x7fc6a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:04.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6bccd6640 1 -- 172.21.15.37:0/3878709247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b815f9c0 msgr2=0x7fc6a00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:04.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6bccd6640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b815f9c0 0x7fc6a00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6bccd6640 1 -- 172.21.15.37:0/3878709247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a0003f60 msgr2=0x7fc6a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:04.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6bccd6640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a0003f60 0x7fc6a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.490+0000 7fc6bccd6640 1 -- 172.21.15.37:0/3878709247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a0003520 con 0x7fc6a0004d40 2024-04-26T20:24:04.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.491+0000 7fc6bccd6640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6a0004d40 0x7fc6a00051c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000aa50 tx=0x7fc6a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcda9b3ef4a5f2b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:04.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.491+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b000a670 con 0x7fc6a0004d40 2024-04-26T20:24:04.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.491+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6b002f070 con 0x7fc6a0004d40 2024-04-26T20:24:04.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.491+0000 7fc6b6ffd640 1 -- 172.21.15.37:0/3878709247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0034470 con 0x7fc6a0004d40 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.491+0000 7fc6be739640 1 -- 172.21.15.37:0/3878709247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6a0004d40 msgr2=0x7fc6a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.491+0000 7fc6be739640 1 --2- 172.21.15.37:0/3878709247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6a0004d40 0x7fc6a00051c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000aa50 tx=0x7fc6a0005830 comp rx=0 tx=0).stop 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.491+0000 7fc6bccd6640 1 -- 172.21.15.37:0/3878709247 reap_dead start 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.492+0000 7fc6be739640 1 -- 172.21.15.37:0/3878709247 shutdown_connections 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.492+0000 7fc6be739640 1 -- 172.21.15.37:0/3878709247 >> 172.21.15.37:0/3878709247 conn(0x7fc6b80d2f80 msgr2=0x7fc6b80d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.492+0000 7fc6be739640 1 -- 172.21.15.37:0/3878709247 shutdown_connections 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.492+0000 7fc6be739640 1 -- 172.21.15.37:0/3878709247 wait complete. 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.493+0000 7fc6be739640 1 Processor -- start 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.493+0000 7fc6be739640 1 -- start start 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6be739640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b8162a50 0x7fc6b81619a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6be739640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b8162e30 0x7fc6b815fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6be739640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6b81602e0 0x7fc6b80d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6be739640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b806e590 con 0x7fc6b8162a50 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6be739640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b806e2b0 con 0x7fc6b81602e0 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6be739640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b806e870 con 0x7fc6b8162e30 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b8162a50 0x7fc6b81619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b8162a50 0x7fc6b81619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59510/0 (socket says 172.21.15.37:59510) 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b7fff640 1 -- 172.21.15.37:0/3750919028 learned_addr learned my addr 172.21.15.37:0/3750919028 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b77fe640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b8162e30 0x7fc6b815fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6bccd6640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6b81602e0 0x7fc6b80d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b7fff640 1 -- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b8162e30 msgr2=0x7fc6b815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b7fff640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b8162e30 0x7fc6b815fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b7fff640 1 -- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6b81602e0 msgr2=0x7fc6b80d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b7fff640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6b81602e0 0x7fc6b80d0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b7fff640 1 -- 172.21.15.37:0/3750919028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b8160920 con 0x7fc6b8162a50 2024-04-26T20:24:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b7fff640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b8162a50 0x7fc6b81619a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0119d0 tx=0x7fc6ac011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:04.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b4ff9640 1 -- 172.21.15.37:0/3750919028 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac00fb50 con 0x7fc6b8162a50 2024-04-26T20:24:04.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b4ff9640 1 -- 172.21.15.37:0/3750919028 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6ac00fcf0 con 0x7fc6b8162a50 2024-04-26T20:24:04.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6be739640 1 -- 172.21.15.37:0/3750919028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b8160be0 con 0x7fc6b8162a50 2024-04-26T20:24:04.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.494+0000 7fc6b4ff9640 1 -- 172.21.15.37:0/3750919028 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac00fea0 con 0x7fc6b8162a50 2024-04-26T20:24:04.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.495+0000 7fc6be739640 1 -- 172.21.15.37:0/3750919028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b81611a0 con 0x7fc6b8162a50 2024-04-26T20:24:04.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.496+0000 7fc6be739640 1 -- 172.21.15.37:0/3750919028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b806a360 con 0x7fc6b8162a50 2024-04-26T20:24:04.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.496+0000 7fc6b4ff9640 1 -- 172.21.15.37:0/3750919028 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fc6ac012040 con 0x7fc6b8162a50 2024-04-26T20:24:04.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.497+0000 7fc6b4ff9640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc6a406cbf0 0x7fc6a406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.497+0000 7fc6b4ff9640 1 -- 172.21.15.37:0/3750919028 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fc6ac08c290 con 0x7fc6b8162a50 2024-04-26T20:24:04.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.497+0000 7fc6b77fe640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc6a406cbf0 0x7fc6a406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:04.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.497+0000 7fc6b77fe640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc6a406cbf0 0x7fc6a406f0b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc6b814d080 tx=0x7fc6a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:04.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.501+0000 7fc6b4ff9640 1 -- 172.21.15.37:0/3750919028 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc6ac019030 con 0x7fc6b8162a50 2024-04-26T20:24:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.684+0000 7fc6be739640 1 -- 172.21.15.37:0/3750919028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc6b8071a70 con 0x7fc6b8162a50 2024-04-26T20:24:04.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.689+0000 7fc6b4ff9640 1 -- 172.21.15.37:0/3750919028 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fc6ac05a700 con 0x7fc6b8162a50 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.691+0000 7fc6be739640 1 -- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc6a406cbf0 msgr2=0x7fc6a406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.691+0000 7fc6be739640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc6a406cbf0 0x7fc6a406f0b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc6b814d080 tx=0x7fc6a8009040 comp rx=0 tx=0).stop 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.691+0000 7fc6be739640 1 -- 172.21.15.37:0/3750919028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b8162a50 msgr2=0x7fc6b81619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.691+0000 7fc6be739640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b8162a50 0x7fc6b81619a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0119d0 tx=0x7fc6ac011e90 comp rx=0 tx=0).stop 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.691+0000 7fc6be739640 1 -- 172.21.15.37:0/3750919028 shutdown_connections 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.691+0000 7fc6be739640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6b81602e0 0x7fc6b80d0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.691+0000 7fc6be739640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc6a406cbf0 0x7fc6a406f0b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.691+0000 7fc6be739640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6b8162e30 0x7fc6b815fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.691+0000 7fc6be739640 1 --2- 172.21.15.37:0/3750919028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b8162a50 0x7fc6b81619a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.691+0000 7fc6be739640 1 -- 172.21.15.37:0/3750919028 >> 172.21.15.37:0/3750919028 conn(0x7fc6b80d2f80 msgr2=0x7fc6b815a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.691+0000 7fc6be739640 1 -- 172.21.15.37:0/3750919028 shutdown_connections 2024-04-26T20:24:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.692+0000 7fc6be739640 1 -- 172.21.15.37:0/3750919028 wait complete. 2024-04-26T20:24:04.704 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:24:04.704 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:24:04.741 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:24:04.786 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:24:04.786 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:24:04.786 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:24:04.801 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=371 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:24:04.801 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:24:04.801 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:24:04.818 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17743 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:24:04.818 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:24:04.818 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:24:04.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=371 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:24:04.857 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:24:04.857 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:24:04.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17743 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:24:04.875 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T20:24:04.875 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:24:04.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.995+0000 7fcb0c493640 1 Processor -- start 2024-04-26T20:24:04.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.995+0000 7fcb0c493640 1 -- start start 2024-04-26T20:24:04.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.996+0000 7fcb0c493640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcb040cfac0 0x7fcb040d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.996+0000 7fcb0c493640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcb040cfea0 0x7fcb040d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.996+0000 7fcb0c493640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb040d0f00 0x7fcb040d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.996+0000 7fcb0c493640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fcb040cf1a0 con 0x7fcb040cfea0 2024-04-26T20:24:04.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.996+0000 7fcb0c493640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcb040ce900 con 0x7fcb040cfac0 2024-04-26T20:24:04.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.996+0000 7fcb0c493640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcb040cebe0 con 0x7fcb040d0f00 2024-04-26T20:24:04.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.996+0000 7fcb09a2e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcb040cfea0 0x7fcb040d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:04.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.996+0000 7fcb09a2e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcb040cfea0 0x7fcb040d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59512/0 (socket says 172.21.15.37:59512) 2024-04-26T20:24:04.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.996+0000 7fcb09a2e640 1 -- 172.21.15.37:0/1182324188 learned_addr learned my addr 172.21.15.37:0/1182324188 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:04.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.997+0000 7fcb0aa30640 1 --2- 172.21.15.37:0/1182324188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb040d0f00 0x7fcb040d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:04.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.997+0000 7fcb0a22f640 1 --2- 172.21.15.37:0/1182324188 >> v2:172.21.15.134:3300/0 conn(0x7fcb040cfac0 0x7fcb040d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:04.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.997+0000 7fcb09a2e640 1 -- 172.21.15.37:0/1182324188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb040d0f00 msgr2=0x7fcb040d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:04.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.997+0000 7fcb09a2e640 1 --2- 172.21.15.37:0/1182324188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb040d0f00 0x7fcb040d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.997+0000 7fcb09a2e640 1 -- 172.21.15.37:0/1182324188 >> v2:172.21.15.134:3300/0 conn(0x7fcb040cfac0 msgr2=0x7fcb040d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:04.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.997+0000 7fcb09a2e640 1 --2- 172.21.15.37:0/1182324188 >> v2:172.21.15.134:3300/0 conn(0x7fcb040cfac0 0x7fcb040d3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.997+0000 7fcb09a2e640 1 -- 172.21.15.37:0/1182324188 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb040ceec0 con 0x7fcb040cfea0 2024-04-26T20:24:05.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.997+0000 7fcb09a2e640 1 --2- 172.21.15.37:0/1182324188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb040cfea0 0x7fcb040d02e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcb040d2f60 tx=0x7fcaf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5eb56b0836d568 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:05.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0922d640 1 -- 172.21.15.37:0/1182324188 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8037070 con 0x7fcb040cfea0 2024-04-26T20:24:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0922d640 1 -- 172.21.15.37:0/1182324188 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf8032440 con 0x7fcb040cfea0 2024-04-26T20:24:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0922d640 1 -- 172.21.15.37:0/1182324188 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8032810 con 0x7fcb040cfea0 2024-04-26T20:24:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0c493640 1 -- 172.21.15.37:0/1182324188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb040cfea0 msgr2=0x7fcb040d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0c493640 1 --2- 172.21.15.37:0/1182324188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb040cfea0 0x7fcb040d02e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcb040d2f60 tx=0x7fcaf802a270 comp rx=0 tx=0).stop 2024-04-26T20:24:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0c493640 1 -- 172.21.15.37:0/1182324188 shutdown_connections 2024-04-26T20:24:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0c493640 1 --2- 172.21.15.37:0/1182324188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb040d0f00 0x7fcb040d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0c493640 1 --2- 172.21.15.37:0/1182324188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb040cfea0 0x7fcb040d02e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0c493640 1 --2- 172.21.15.37:0/1182324188 >> v2:172.21.15.134:3300/0 conn(0x7fcb040cfac0 0x7fcb040d3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0c493640 1 -- 172.21.15.37:0/1182324188 >> 172.21.15.37:0/1182324188 conn(0x7fcb040d3000 msgr2=0x7fcb040d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0c493640 1 -- 172.21.15.37:0/1182324188 shutdown_connections 2024-04-26T20:24:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.998+0000 7fcb0c493640 1 -- 172.21.15.37:0/1182324188 wait complete. 2024-04-26T20:24:05.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.999+0000 7fcb0c493640 1 Processor -- start 2024-04-26T20:24:05.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:04.999+0000 7fcb0c493640 1 -- start start 2024-04-26T20:24:05.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.000+0000 7fcb0c493640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb040cfac0 0x7fcb0415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:05.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.000+0000 7fcb0c493640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb040cfea0 0x7fcb040b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:05.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.000+0000 7fcb0c493640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb040d0f00 0x7fcb040c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:05.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.000+0000 7fcb0c493640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040d1a70 con 0x7fcb040cfac0 2024-04-26T20:24:05.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.000+0000 7fcb0c493640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040d18f0 con 0x7fcb040cfea0 2024-04-26T20:24:05.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.000+0000 7fcb0c493640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcb040d1bf0 con 0x7fcb040d0f00 2024-04-26T20:24:05.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.000+0000 7fcb0a22f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb040cfac0 0x7fcb0415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.000+0000 7fcb0a22f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb040cfac0 0x7fcb0415d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59518/0 (socket says 172.21.15.37:59518) 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.000+0000 7fcb0a22f640 1 -- 172.21.15.37:0/1457370896 learned_addr learned my addr 172.21.15.37:0/1457370896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb09a2e640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb040cfea0 0x7fcb040b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb0aa30640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb040d0f00 0x7fcb040c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb0a22f640 1 -- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb040d0f00 msgr2=0x7fcb040c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb0a22f640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb040d0f00 0x7fcb040c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb0a22f640 1 -- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb040cfea0 msgr2=0x7fcb040b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb0a22f640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb040cfea0 0x7fcb040b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb0a22f640 1 -- 172.21.15.37:0/1457370896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb041f3580 con 0x7fcb040cfac0 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb09a2e640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb040cfea0 0x7fcb040b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb0a22f640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb040cfac0 0x7fcb0415d6d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00c7f0 tx=0x7fcafc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb02ffd640 1 -- 172.21.15.37:0/1457370896 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc015070 con 0x7fcb040cfac0 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb02ffd640 1 -- 172.21.15.37:0/1457370896 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcafc004240 con 0x7fcb040cfac0 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb0c493640 1 -- 172.21.15.37:0/1457370896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb041f3860 con 0x7fcb040cfac0 2024-04-26T20:24:05.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb02ffd640 1 -- 172.21.15.37:0/1457370896 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc007620 con 0x7fcb040cfac0 2024-04-26T20:24:05.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.001+0000 7fcb0c493640 1 -- 172.21.15.37:0/1457370896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb041f3e20 con 0x7fcb040cfac0 2024-04-26T20:24:05.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.002+0000 7fcb0c493640 1 -- 172.21.15.37:0/1457370896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb040b4920 con 0x7fcb040cfac0 2024-04-26T20:24:05.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.003+0000 7fcb02ffd640 1 -- 172.21.15.37:0/1457370896 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fcafc0077c0 con 0x7fcb040cfac0 2024-04-26T20:24:05.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.003+0000 7fcb02ffd640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaf006cbc0 0x7fcaf006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:05.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.003+0000 7fcb02ffd640 1 -- 172.21.15.37:0/1457370896 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fcafc08c250 con 0x7fcb040cfac0 2024-04-26T20:24:05.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.003+0000 7fcb09a2e640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaf006cbc0 0x7fcaf006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.004+0000 7fcb09a2e640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaf006cbc0 0x7fcaf006f080 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8034000 tx=0x7fcaf802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:05.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.008+0000 7fcb02ffd640 1 -- 172.21.15.37:0/1457370896 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcafc05a310 con 0x7fcb040cfac0 2024-04-26T20:24:05.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.232+0000 7fcb0c493640 1 -- 172.21.15.37:0/1457370896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb04069670 con 0x7fcb040cfac0 2024-04-26T20:24:05.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.232+0000 7fcb02ffd640 1 -- 172.21.15.37:0/1457370896 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3875 (secure 0 0 0) 0x7fcafc059c10 con 0x7fcb040cfac0 2024-04-26T20:24:05.234 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:05.234 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/4166977617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4166977617},{"type":"v1","addr":"172.21.15.134:6836","nonce":4166977617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3245561282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3245561282},{"type":"v1","addr":"172.21.15.134:6837","nonce":3245561282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3853963851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3853963851},{"type":"v1","addr":"172.21.15.37:6835","nonce":3853963851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/1336518193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1336518193},{"type":"v1","addr":"172.21.15.37:6834","nonce":1336518193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-04-26T20:24:05.234 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 18 2024-04-26T20:24:05.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.234+0000 7fcb0c493640 1 -- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaf006cbc0 msgr2=0x7fcaf006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:05.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.234+0000 7fcb0c493640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaf006cbc0 0x7fcaf006f080 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8034000 tx=0x7fcaf802e000 comp rx=0 tx=0).stop 2024-04-26T20:24:05.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.234+0000 7fcb0c493640 1 -- 172.21.15.37:0/1457370896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb040cfac0 msgr2=0x7fcb0415d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:05.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.234+0000 7fcb0c493640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb040cfac0 0x7fcb0415d6d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00c7f0 tx=0x7fcafc00ccb0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.235+0000 7fcb0c493640 1 -- 172.21.15.37:0/1457370896 shutdown_connections 2024-04-26T20:24:05.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.235+0000 7fcb0c493640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcaf006cbc0 0x7fcaf006f080 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.235+0000 7fcb0c493640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb040d0f00 0x7fcb040c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.235+0000 7fcb0c493640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb040cfea0 0x7fcb040b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.235+0000 7fcb0c493640 1 --2- 172.21.15.37:0/1457370896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb040cfac0 0x7fcb0415d6d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.235+0000 7fcb0c493640 1 -- 172.21.15.37:0/1457370896 >> 172.21.15.37:0/1457370896 conn(0x7fcb040d3000 msgr2=0x7fcb040cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:05.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.235+0000 7fcb0c493640 1 -- 172.21.15.37:0/1457370896 shutdown_connections 2024-04-26T20:24:05.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.235+0000 7fcb0c493640 1 -- 172.21.15.37:0/1457370896 wait complete. 2024-04-26T20:24:05.247 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:24:05.247 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T20:24:05.247 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:24:05.247 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:24:05.247 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:24:05.248 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T20:24:05.248 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:24:05.248 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:24:05.248 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:24:05.248 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T20:24:05.248 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:24:05.248 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:24:05.249 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:24:05.249 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T20:24:05.249 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:24:05.249 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:24:05.257 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:24:05.257 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:24:05.258 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:24:05.258 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:24:05.261 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:24:05.261 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:24:05.262 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:24:05.262 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:24:05.300 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:24:05.300 DEBUG:teuthology.parallel:result is None 2024-04-26T20:24:05.301 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:24:05.301 DEBUG:teuthology.parallel:result is None 2024-04-26T20:24:05.304 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:24:05.304 DEBUG:teuthology.parallel:result is None 2024-04-26T20:24:05.304 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:24:05.305 DEBUG:teuthology.parallel:result is None 2024-04-26T20:24:05.305 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T20:24:05.375 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-26T20:24:05.376 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-26T20:24:05.384 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-04-26T20:24:05.384 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-04-26T20:24:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.410+0000 7f9def3c6640 1 Processor -- start 2024-04-26T20:24:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.410+0000 7f9def3c6640 1 -- start start 2024-04-26T20:24:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.411+0000 7f9def3c6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9de8162b40 0x7f9de80d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.411+0000 7f9def3c6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9de80d3c80 0x7f9de80d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.411+0000 7f9def3c6640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de80d0210 con 0x7f9de80d3820 2024-04-26T20:24:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.411+0000 7f9def3c6640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f9de80d04f0 con 0x7f9de80d3c80 2024-04-26T20:24:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.411+0000 7f9def3c6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9de80d07d0 con 0x7f9de8162b40 2024-04-26T20:24:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.411+0000 7f9dec961640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9de80d3820 0x7f9de80d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33278/0 (socket says 172.21.15.37:33278) 2024-04-26T20:24:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.411+0000 7f9dec961640 1 -- 172.21.15.37:0/2372184461 learned_addr learned my addr 172.21.15.37:0/2372184461 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.411+0000 7f9ded963640 1 --2- 172.21.15.37:0/2372184461 >> v2:172.21.15.37:3300/0 conn(0x7f9de80d3c80 0x7f9de80d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.412+0000 7f9ded162640 1 --2- 172.21.15.37:0/2372184461 >> v2:172.21.15.134:3300/0 conn(0x7f9de8162b40 0x7f9de80d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.412+0000 7f9ded963640 1 -- 172.21.15.37:0/2372184461 >> v2:172.21.15.134:3300/0 conn(0x7f9de8162b40 msgr2=0x7f9de80d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.412+0000 7f9ded963640 1 --2- 172.21.15.37:0/2372184461 >> v2:172.21.15.134:3300/0 conn(0x7f9de8162b40 0x7f9de80d5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.412+0000 7f9ded963640 1 -- 172.21.15.37:0/2372184461 >> v1:172.21.15.37:6789/0 conn(0x7f9de80d3820 legacy=0x7f9de80d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.412+0000 7f9ded963640 1 -- 172.21.15.37:0/2372184461 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de80d0ab0 con 0x7f9de80d3c80 2024-04-26T20:24:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.412+0000 7f9ddffff640 1 -- 172.21.15.37:0/2372184461 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185358300 0 0) 0x7f9de80d0210 con 0x7f9de80d3820 2024-04-26T20:24:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9ded963640 1 --2- 172.21.15.37:0/2372184461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de80d3c80 0x7f9de80d1f60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9de80d2910 tx=0x7f9de002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32951e24ed885ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9ddffff640 1 -- 172.21.15.37:0/2372184461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0007740 con 0x7f9de80d3c80 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9ddffff640 1 -- 172.21.15.37:0/2372184461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de00078e0 con 0x7f9de80d3c80 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9ddffff640 1 -- 172.21.15.37:0/2372184461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0007cb0 con 0x7f9de80d3c80 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9def3c6640 1 -- 172.21.15.37:0/2372184461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de80d3c80 msgr2=0x7f9de80d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9def3c6640 1 --2- 172.21.15.37:0/2372184461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de80d3c80 0x7f9de80d1f60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9de80d2910 tx=0x7f9de002a710 comp rx=0 tx=0).stop 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9def3c6640 1 -- 172.21.15.37:0/2372184461 shutdown_connections 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9def3c6640 1 --2- 172.21.15.37:0/2372184461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de80d3c80 0x7f9de80d1f60 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9def3c6640 1 --2- 172.21.15.37:0/2372184461 >> v2:172.21.15.134:3300/0 conn(0x7f9de8162b40 0x7f9de80d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9def3c6640 1 -- 172.21.15.37:0/2372184461 >> 172.21.15.37:0/2372184461 conn(0x7f9de809e550 msgr2=0x7f9de80d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9def3c6640 1 -- 172.21.15.37:0/2372184461 shutdown_connections 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.413+0000 7f9def3c6640 1 -- 172.21.15.37:0/2372184461 wait complete. 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.414+0000 7f9def3c6640 1 Processor -- start 2024-04-26T20:24:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.415+0000 7f9def3c6640 1 -- start start 2024-04-26T20:24:05.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.415+0000 7f9def3c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de80d3820 0x7f9de815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:05.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.415+0000 7f9def3c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de80d3c80 0x7f9de806d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:05.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.415+0000 7f9def3c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de8162b40 0x7f9de806db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:05.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.415+0000 7f9def3c6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9de80b5bb0 con 0x7f9de8162b40 2024-04-26T20:24:05.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.415+0000 7f9def3c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9de80d4110 con 0x7f9de80d3c80 2024-04-26T20:24:05.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.415+0000 7f9def3c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9de80d6110 con 0x7f9de80d3820 2024-04-26T20:24:05.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.415+0000 7f9ded963640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de8162b40 0x7f9de806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.415+0000 7f9ded963640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de8162b40 0x7f9de806db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59580/0 (socket says 172.21.15.37:59580) 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.415+0000 7f9ded963640 1 -- 172.21.15.37:0/3635991048 learned_addr learned my addr 172.21.15.37:0/3635991048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9dec961640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de80d3c80 0x7f9de806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9ded162640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de80d3820 0x7f9de815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9ded963640 1 -- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de80d3820 msgr2=0x7f9de815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9ded963640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de80d3820 0x7f9de815d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9ded963640 1 -- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de80d3c80 msgr2=0x7f9de806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9ded963640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de80d3c80 0x7f9de806d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9ded963640 1 -- 172.21.15.37:0/3635991048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de80c9800 con 0x7f9de8162b40 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9dec961640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de80d3c80 0x7f9de806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9ded963640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de8162b40 0x7f9de806db70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009e80 tx=0x7f9de0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9dddffb640 1 -- 172.21.15.37:0/3635991048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0040070 con 0x7f9de8162b40 2024-04-26T20:24:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9dddffb640 1 -- 172.21.15.37:0/3635991048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de0031c40 con 0x7f9de8162b40 2024-04-26T20:24:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9dddffb640 1 -- 172.21.15.37:0/3635991048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0003b90 con 0x7f9de8162b40 2024-04-26T20:24:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9def3c6640 1 -- 172.21.15.37:0/3635991048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de80c9ae0 con 0x7f9de8162b40 2024-04-26T20:24:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.416+0000 7f9def3c6640 1 -- 172.21.15.37:0/3635991048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de80ca0a0 con 0x7f9de8162b40 2024-04-26T20:24:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.417+0000 7f9def3c6640 1 -- 172.21.15.37:0/3635991048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de806a360 con 0x7f9de8162b40 2024-04-26T20:24:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.418+0000 7f9dddffb640 1 -- 172.21.15.37:0/3635991048 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f9de0039040 con 0x7f9de8162b40 2024-04-26T20:24:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.418+0000 7f9dddffb640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dcc06cbc0 0x7f9dcc06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.418+0000 7f9dddffb640 1 -- 172.21.15.37:0/3635991048 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f9de00ad150 con 0x7f9de8162b40 2024-04-26T20:24:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.418+0000 7f9ded162640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dcc06cbc0 0x7f9dcc06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.419+0000 7f9ded162640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dcc06cbc0 0x7f9dcc06f080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9de81f84e0 tx=0x7f9dd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:05.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.422+0000 7f9dddffb640 1 -- 172.21.15.37:0/3635991048 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9de007b420 con 0x7f9de8162b40 2024-04-26T20:24:05.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:05.621+0000 7f9def3c6640 1 -- 172.21.15.37:0/3635991048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9de80d3120 con 0x7f9de8162b40 2024-04-26T20:24:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.378+0000 7f9dddffb640 1 -- 172.21.15.37:0/3635991048 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) ==== 97+0+0 (secure 0 0 0) 0x7f9de007ad20 con 0x7f9de8162b40 2024-04-26T20:24:06.379 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-04-26T20:24:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 -- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dcc06cbc0 msgr2=0x7f9dcc06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dcc06cbc0 0x7f9dcc06f080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9de81f84e0 tx=0x7f9dd8008040 comp rx=0 tx=0).stop 2024-04-26T20:24:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 -- 172.21.15.37:0/3635991048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de8162b40 msgr2=0x7f9de806db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de8162b40 0x7f9de806db70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009e80 tx=0x7f9de0004a40 comp rx=0 tx=0).stop 2024-04-26T20:24:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 -- 172.21.15.37:0/3635991048 shutdown_connections 2024-04-26T20:24:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de8162b40 0x7f9de806db70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de80d3c80 0x7f9de806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dcc06cbc0 0x7f9dcc06f080 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 --2- 172.21.15.37:0/3635991048 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de80d3820 0x7f9de815d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 -- 172.21.15.37:0/3635991048 >> 172.21.15.37:0/3635991048 conn(0x7f9de809e550 msgr2=0x7f9de80b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 -- 172.21.15.37:0/3635991048 shutdown_connections 2024-04-26T20:24:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.382+0000 7f9def3c6640 1 -- 172.21.15.37:0/3635991048 wait complete. 2024-04-26T20:24:06.397 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T20:24:06.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.503+0000 7f912b485640 1 Processor -- start 2024-04-26T20:24:06.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.503+0000 7f912b485640 1 -- start start 2024-04-26T20:24:06.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.504+0000 7f912b485640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912406f790 0x7f9124074030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.504+0000 7f912b485640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9124160d70 0x7f91241611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.504+0000 7f912b485640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f912415d780 0x7f912415dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.504+0000 7f912b485640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f9124161710 con 0x7f9124160d70 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.504+0000 7f912b485640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9124161b80 con 0x7f912415d780 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.504+0000 7f912b485640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f91240b4400 con 0x7f912406f790 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.504+0000 7f9128a20640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9124160d70 0x7f91241611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.504+0000 7f9128a20640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9124160d70 0x7f91241611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43628/0 (socket says 172.21.15.37:43628) 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.504+0000 7f9128a20640 1 -- 172.21.15.37:0/2462181122 learned_addr learned my addr 172.21.15.37:0/2462181122 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.505+0000 7f9129221640 1 --2- 172.21.15.37:0/2462181122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912406f790 0x7f9124074030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.505+0000 7f9129a22640 1 --2- 172.21.15.37:0/2462181122 >> v2:172.21.15.134:3300/0 conn(0x7f912415d780 0x7f912415dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.505+0000 7f9128a20640 1 -- 172.21.15.37:0/2462181122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912406f790 msgr2=0x7f9124074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.505+0000 7f9128a20640 1 --2- 172.21.15.37:0/2462181122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912406f790 0x7f9124074030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.505+0000 7f9128a20640 1 -- 172.21.15.37:0/2462181122 >> v2:172.21.15.134:3300/0 conn(0x7f912415d780 msgr2=0x7f912415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.505+0000 7f9128a20640 1 --2- 172.21.15.37:0/2462181122 >> v2:172.21.15.134:3300/0 conn(0x7f912415d780 0x7f912415dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.505+0000 7f9128a20640 1 -- 172.21.15.37:0/2462181122 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9124074570 con 0x7f9124160d70 2024-04-26T20:24:06.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.505+0000 7f9129a22640 1 --2- 172.21.15.37:0/2462181122 >> v2:172.21.15.134:3300/0 conn(0x7f912415d780 0x7f912415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:06.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.505+0000 7f9128a20640 1 --2- 172.21.15.37:0/2462181122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9124160d70 0x7f91241611d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9124157df0 tx=0x7f911802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8683947e37b86c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:06.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f9123fff640 1 -- 172.21.15.37:0/2462181122 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118037070 con 0x7f9124160d70 2024-04-26T20:24:06.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f9123fff640 1 -- 172.21.15.37:0/2462181122 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9118032440 con 0x7f9124160d70 2024-04-26T20:24:06.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f9123fff640 1 -- 172.21.15.37:0/2462181122 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118032810 con 0x7f9124160d70 2024-04-26T20:24:06.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f912b485640 1 -- 172.21.15.37:0/2462181122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9124160d70 msgr2=0x7f91241611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f912b485640 1 --2- 172.21.15.37:0/2462181122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9124160d70 0x7f91241611d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9124157df0 tx=0x7f911802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f912b485640 1 -- 172.21.15.37:0/2462181122 shutdown_connections 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f912b485640 1 --2- 172.21.15.37:0/2462181122 >> v2:172.21.15.134:3300/0 conn(0x7f912415d780 0x7f912415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f912b485640 1 --2- 172.21.15.37:0/2462181122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9124160d70 0x7f91241611d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f912b485640 1 --2- 172.21.15.37:0/2462181122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912406f790 0x7f9124074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f912b485640 1 -- 172.21.15.37:0/2462181122 >> 172.21.15.37:0/2462181122 conn(0x7f91240d0d50 msgr2=0x7f91240d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f912b485640 1 -- 172.21.15.37:0/2462181122 shutdown_connections 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.506+0000 7f912b485640 1 -- 172.21.15.37:0/2462181122 wait complete. 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.507+0000 7f912b485640 1 Processor -- start 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.508+0000 7f912b485640 1 -- start start 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.508+0000 7f912b485640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912406f790 0x7f91241e6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.508+0000 7f912b485640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f912415d780 0x7f91240d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.508+0000 7f912b485640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9124160d70 0x7f9124157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.508+0000 7f912b485640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f91240732d0 con 0x7f912415d780 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.508+0000 7f912b485640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9124072ff0 con 0x7f912406f790 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.508+0000 7f912b485640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f91240735b0 con 0x7f9124160d70 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.508+0000 7f9128a20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f912415d780 0x7f91240d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.508+0000 7f9128a20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f912415d780 0x7f91240d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43638/0 (socket says 172.21.15.37:43638) 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.508+0000 7f9128a20640 1 -- 172.21.15.37:0/3558935714 learned_addr learned my addr 172.21.15.37:0/3558935714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.508+0000 7f9129221640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912406f790 0x7f91241e6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f9129a22640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9124160d70 0x7f9124157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f9128a20640 1 -- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9124160d70 msgr2=0x7f9124157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f9128a20640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9124160d70 0x7f9124157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f9128a20640 1 -- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912406f790 msgr2=0x7f91241e6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f9128a20640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912406f790 0x7f91241e6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f9128a20640 1 -- 172.21.15.37:0/3558935714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91240cf360 con 0x7f912415d780 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f9128a20640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f912415d780 0x7f91240d0400 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f911802a220 tx=0x7f91180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f9121ffb640 1 -- 172.21.15.37:0/3558935714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118037040 con 0x7f912415d780 2024-04-26T20:24:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f9121ffb640 1 -- 172.21.15.37:0/3558935714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9118042070 con 0x7f912415d780 2024-04-26T20:24:06.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f912b485640 1 -- 172.21.15.37:0/3558935714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91240cf640 con 0x7f912415d780 2024-04-26T20:24:06.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f9121ffb640 1 -- 172.21.15.37:0/3558935714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118030bb0 con 0x7f912415d780 2024-04-26T20:24:06.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.509+0000 7f912b485640 1 -- 172.21.15.37:0/3558935714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91240cfc00 con 0x7f912415d780 2024-04-26T20:24:06.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.510+0000 7f912b485640 1 -- 172.21.15.37:0/3558935714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912406a360 con 0x7f912415d780 2024-04-26T20:24:06.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.511+0000 7f9121ffb640 1 -- 172.21.15.37:0/3558935714 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f9118047460 con 0x7f912415d780 2024-04-26T20:24:06.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.511+0000 7f9121ffb640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f911006cbf0 0x7f911006f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.511+0000 7f9121ffb640 1 -- 172.21.15.37:0/3558935714 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f91180ac790 con 0x7f912415d780 2024-04-26T20:24:06.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.511+0000 7f9129221640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f911006cbf0 0x7f911006f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.512+0000 7f9129221640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f911006cbf0 0x7f911006f0b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f911c00a9b0 tx=0x7f911c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:06.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.516+0000 7f9121ffb640 1 -- 172.21.15.37:0/3558935714 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f911807a530 con 0x7f912415d780 2024-04-26T20:24:06.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.773+0000 7f912b485640 1 -- 172.21.15.37:0/3558935714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f912406ff40 con 0x7f912415d780 2024-04-26T20:24:06.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.774+0000 7f9121ffb640 1 -- 172.21.15.37:0/3558935714 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f911807e830 con 0x7f912415d780 2024-04-26T20:24:06.775 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:06.775 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:24:06.775 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-04-26T20:24:06.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.775 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-04-26T20:24:06.775 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxmMZ1xLRAA25Lk5iPjTAlo8+3CnWPavg==", 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxmSCb5OBAA1qn4Xl59ur/wJDapWSLZ9A==", 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-26T20:24:06.776 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxmxYZlNBAAqm/foyZ1j157Yi/vkOspOg==", 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.777 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCkDCxmx942BBAAJyAAYb/ZMcI0ZYLluqttZw==", 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCnDCxmuavuHhAAHPS8O9ExPpvdHT10wJDHrw==", 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.778 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCqDCxmHDa3HRAAVysGtbYHxPLpHx3kdS1EKw==", 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.779 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCtDCxmwplYHBAAxzI1MLVpn8kK1B27n/ZBoA==", 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCwDCxmlem9GxAAR6h6rw0owBPPUT/Eyh26bQ==", 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.780 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC2DCxmqKnHHhAAkvIA0VP38Y2RYBKMdZFuZA==", 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC5DCxmGU7hHBAAAXOqhBDJW/ZiUYlY4dv/DA==", 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.781 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC8DCxmF7uHGhAAKioy1jI7alaj7WSVHJ1nKA==", 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.782 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC/DCxmKk8wGRAAaghSXnN4nUlxzhkYGBzZig==", 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCkDCxmQTcnCBAAFh1Ew2ihHFtREYikzwF+Dw==", 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T20:24:06.783 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCkDCxmXGS+EBAAvWfY/Xt9yfBLUr3jhc44cA==", 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-04-26T20:24:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCkDCxmUdq2DRAAq/Hml2I/z0bkKXjN9jIeDg==", 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCkDCxmtrYnFhAADmyeYHIoG22nzUEozY0R9Q==", 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-26T20:24:06.785 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxmazwnERAAfhEVfmWlGnPrPPmMWxbOKQ==", 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-04-26T20:24:06.786 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxm5VtOKhAA9ZJhTDAGuUn+Un63aSPDbQ==", 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCjDCxmUhBnJRAAGsrfntQyYUoHJlj2M2OCVA==", 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.787 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:06.788 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T20:24:06.788 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:24:06.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 -- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f911006cbf0 msgr2=0x7f911006f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:06.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f911006cbf0 0x7f911006f0b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f911c00a9b0 tx=0x7f911c008040 comp rx=0 tx=0).stop 2024-04-26T20:24:06.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 -- 172.21.15.37:0/3558935714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f912415d780 msgr2=0x7f91240d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:06.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f912415d780 0x7f91240d0400 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f911802a220 tx=0x7f91180046c0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 -- 172.21.15.37:0/3558935714 shutdown_connections 2024-04-26T20:24:06.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f911006cbf0 0x7f911006f0b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9124160d70 0x7f9124157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f912415d780 0x7f91240d0400 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 --2- 172.21.15.37:0/3558935714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912406f790 0x7f91241e6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 -- 172.21.15.37:0/3558935714 >> 172.21.15.37:0/3558935714 conn(0x7f91240d0d50 msgr2=0x7f91240cdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:06.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 -- 172.21.15.37:0/3558935714 shutdown_connections 2024-04-26T20:24:06.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.776+0000 7f912b485640 1 -- 172.21.15.37:0/3558935714 wait complete. 2024-04-26T20:24:06.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:24:06.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.891+0000 7f8d79046640 1 Processor -- start 2024-04-26T20:24:06.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.891+0000 7f8d79046640 1 -- start start 2024-04-26T20:24:06.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d79046640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8d74162e40 0x7f8d740d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d79046640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8d740b2050 0x7f8d740cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d79046640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d740cfd70 0x7f8d740ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d79046640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f8d74163220 con 0x7f8d74162e40 2024-04-26T20:24:06.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d79046640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f8d740d0eb0 con 0x7f8d740b2050 2024-04-26T20:24:06.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d79046640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8d740d0180 con 0x7f8d740cfd70 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d72d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8d74162e40 0x7f8d740d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d72d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8d74162e40 0x7f8d740d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43650/0 (socket says 172.21.15.37:43650) 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d72d9d640 1 -- 172.21.15.37:0/2458654574 learned_addr learned my addr 172.21.15.37:0/2458654574 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d6bfff640 1 --2- 172.21.15.37:0/2458654574 >> v2:172.21.15.134:3300/0 conn(0x7f8d740b2050 0x7f8d740cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d7359e640 1 --2- 172.21.15.37:0/2458654574 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d740cfd70 0x7f8d740ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d72d9d640 1 -- 172.21.15.37:0/2458654574 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d740cfd70 msgr2=0x7f8d740ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d72d9d640 1 --2- 172.21.15.37:0/2458654574 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d740cfd70 0x7f8d740ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d72d9d640 1 -- 172.21.15.37:0/2458654574 >> v2:172.21.15.134:3300/0 conn(0x7f8d740b2050 msgr2=0x7f8d740cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d72d9d640 1 --2- 172.21.15.37:0/2458654574 >> v2:172.21.15.134:3300/0 conn(0x7f8d740b2050 0x7f8d740cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.892+0000 7f8d72d9d640 1 -- 172.21.15.37:0/2458654574 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d740b1d70 con 0x7f8d74162e40 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.893+0000 7f8d72d9d640 1 --2- 172.21.15.37:0/2458654574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d74162e40 0x7f8d740d1340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8d74069bf0 tx=0x7f8d6402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad76053d08ce79c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.893+0000 7f8d7259c640 1 -- 172.21.15.37:0/2458654574 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64037070 con 0x7f8d74162e40 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.893+0000 7f8d7259c640 1 -- 172.21.15.37:0/2458654574 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d64032440 con 0x7f8d74162e40 2024-04-26T20:24:06.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.893+0000 7f8d7259c640 1 -- 172.21.15.37:0/2458654574 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64032810 con 0x7f8d74162e40 2024-04-26T20:24:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.894+0000 7f8d79046640 1 -- 172.21.15.37:0/2458654574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d74162e40 msgr2=0x7f8d740d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.894+0000 7f8d79046640 1 --2- 172.21.15.37:0/2458654574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d74162e40 0x7f8d740d1340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8d74069bf0 tx=0x7f8d6402a140 comp rx=0 tx=0).stop 2024-04-26T20:24:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.894+0000 7f8d79046640 1 -- 172.21.15.37:0/2458654574 shutdown_connections 2024-04-26T20:24:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.894+0000 7f8d79046640 1 --2- 172.21.15.37:0/2458654574 >> v2:172.21.15.134:3300/0 conn(0x7f8d740b2050 0x7f8d740cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.894+0000 7f8d79046640 1 --2- 172.21.15.37:0/2458654574 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d740cfd70 0x7f8d740ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.894+0000 7f8d79046640 1 --2- 172.21.15.37:0/2458654574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d74162e40 0x7f8d740d1340 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.894+0000 7f8d79046640 1 -- 172.21.15.37:0/2458654574 >> 172.21.15.37:0/2458654574 conn(0x7f8d7409ba50 msgr2=0x7f8d7409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.894+0000 7f8d79046640 1 -- 172.21.15.37:0/2458654574 shutdown_connections 2024-04-26T20:24:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.894+0000 7f8d79046640 1 -- 172.21.15.37:0/2458654574 wait complete. 2024-04-26T20:24:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.895+0000 7f8d79046640 1 Processor -- start 2024-04-26T20:24:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.895+0000 7f8d79046640 1 -- start start 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d79046640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d740b2050 0x7f8d740c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d79046640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d740cfd70 0x7f8d7415c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d79046640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d74162e40 0x7f8d740c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d79046640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8d740d1f20 con 0x7f8d74162e40 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d79046640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d740d1c40 con 0x7f8d740b2050 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d79046640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8d740b24e0 con 0x7f8d740cfd70 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d7359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d74162e40 0x7f8d740c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d7359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d74162e40 0x7f8d740c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43660/0 (socket says 172.21.15.37:43660) 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d7359e640 1 -- 172.21.15.37:0/3312433546 learned_addr learned my addr 172.21.15.37:0/3312433546 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d6bfff640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d740cfd70 0x7f8d7415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d72d9d640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d740b2050 0x7f8d740c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d7359e640 1 -- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d740cfd70 msgr2=0x7f8d7415c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d7359e640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d740cfd70 0x7f8d7415c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d7359e640 1 -- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d740b2050 msgr2=0x7f8d740c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:06.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d7359e640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d740b2050 0x7f8d740c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d7359e640 1 -- 172.21.15.37:0/3312433546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d740c7260 con 0x7f8d74162e40 2024-04-26T20:24:06.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.896+0000 7f8d72d9d640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d740b2050 0x7f8d740c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:06.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.897+0000 7f8d7359e640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d74162e40 0x7f8d740c7fa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00cd70 tx=0x7f8d6c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:06.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.897+0000 7f8d6b7fe640 1 -- 172.21.15.37:0/3312433546 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c003b90 con 0x7f8d74162e40 2024-04-26T20:24:06.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.897+0000 7f8d6b7fe640 1 -- 172.21.15.37:0/3312433546 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6c003d30 con 0x7f8d74162e40 2024-04-26T20:24:06.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.897+0000 7f8d79046640 1 -- 172.21.15.37:0/3312433546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d740c7540 con 0x7f8d74162e40 2024-04-26T20:24:06.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.897+0000 7f8d6b7fe640 1 -- 172.21.15.37:0/3312433546 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c004240 con 0x7f8d74162e40 2024-04-26T20:24:06.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.897+0000 7f8d79046640 1 -- 172.21.15.37:0/3312433546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d740c7b00 con 0x7f8d74162e40 2024-04-26T20:24:06.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.898+0000 7f8d79046640 1 -- 172.21.15.37:0/3312433546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d7406a360 con 0x7f8d74162e40 2024-04-26T20:24:06.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.898+0000 7f8d6b7fe640 1 -- 172.21.15.37:0/3312433546 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f8d6c0043e0 con 0x7f8d74162e40 2024-04-26T20:24:06.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.899+0000 7f8d6b7fe640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8d5406cbc0 0x7f8d5406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.899+0000 7f8d6b7fe640 1 -- 172.21.15.37:0/3312433546 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f8d6c08c7e0 con 0x7f8d74162e40 2024-04-26T20:24:06.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.899+0000 7f8d72d9d640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8d5406cbc0 0x7f8d5406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.899+0000 7f8d72d9d640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8d5406cbc0 0x7f8d5406f080 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8d64034000 tx=0x7f8d6402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:06.903+0000 7f8d6b7fe640 1 -- 172.21.15.37:0/3312433546 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8d6c05ab00 con 0x7f8d74162e40 2024-04-26T20:24:07.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.087+0000 7f8d79046640 1 -- 172.21.15.37:0/3312433546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d740717c0 con 0x7f8d74162e40 2024-04-26T20:24:07.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.088+0000 7f8d6b7fe640 1 -- 172.21.15.37:0/3312433546 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+12903 (secure 0 0 0) 0x7f8d6c05a400 con 0x7f8d74162e40 2024-04-26T20:24:07.089 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:07.089 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":42,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:24:06.390532+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:24:07.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 -- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8d5406cbc0 msgr2=0x7f8d5406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8d5406cbc0 0x7f8d5406f080 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8d64034000 tx=0x7f8d6402e000 comp rx=0 tx=0).stop 2024-04-26T20:24:07.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 -- 172.21.15.37:0/3312433546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d74162e40 msgr2=0x7f8d740c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d74162e40 0x7f8d740c7fa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00cd70 tx=0x7f8d6c00a660 comp rx=0 tx=0).stop 2024-04-26T20:24:07.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 -- 172.21.15.37:0/3312433546 shutdown_connections 2024-04-26T20:24:07.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d74162e40 0x7f8d740c7fa0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8d5406cbc0 0x7f8d5406f080 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d740cfd70 0x7f8d7415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 --2- 172.21.15.37:0/3312433546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d740b2050 0x7f8d740c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 -- 172.21.15.37:0/3312433546 >> 172.21.15.37:0/3312433546 conn(0x7f8d7409ba50 msgr2=0x7f8d740cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:07.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 -- 172.21.15.37:0/3312433546 shutdown_connections 2024-04-26T20:24:07.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.090+0000 7f8d79046640 1 -- 172.21.15.37:0/3312433546 wait complete. 2024-04-26T20:24:07.102 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:24:07.103 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:24:07.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.206+0000 7fc4d2be9640 1 Processor -- start 2024-04-26T20:24:07.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.206+0000 7fc4d2be9640 1 -- start start 2024-04-26T20:24:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4d2be9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc4cc15f9c0 0x7fc4cc0d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4d2be9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4cc0cf990 0x7fc4cc0cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4d2be9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4cc0b2550 con 0x7fc4cc163380 2024-04-26T20:24:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4d2be9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc4cc0d0310 con 0x7fc4cc15f9c0 2024-04-26T20:24:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4d2be9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc4cc0b3760 con 0x7fc4cc0cf990 2024-04-26T20:24:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4d1186640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc4cc163380 0x7fc4cc163810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59698/0 (socket says 172.21.15.37:59698) 2024-04-26T20:24:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4d1186640 1 -- 172.21.15.37:0/2565709775 learned_addr learned my addr 172.21.15.37:0/2565709775 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 939047595 0 0) 0x7fc4c4001490 con 0x7fc4cc163380 2024-04-26T20:24:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cbfff640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4cc0cf990 0x7fc4cc0cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4d0985640 1 --2- 172.21.15.37:0/2565709775 >> v2:172.21.15.134:3300/0 conn(0x7fc4cc15f9c0 0x7fc4cc0d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4bc003520 con 0x7fc4cc163380 2024-04-26T20:24:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 623282042 0 0) 0x7fc4bc003520 con 0x7fc4cc163380 2024-04-26T20:24:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4cc0cf990 msgr2=0x7fc4cc0cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4cc0cf990 0x7fc4cc0cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 >> v2:172.21.15.134:3300/0 conn(0x7fc4cc15f9c0 msgr2=0x7fc4cc0d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 --2- 172.21.15.37:0/2565709775 >> v2:172.21.15.134:3300/0 conn(0x7fc4cc15f9c0 0x7fc4cc0d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc06dcf0 con 0x7fc4cc163380 2024-04-26T20:24:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fc4c4001bc0 con 0x7fc4cc163380 2024-04-26T20:24:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 >> v1:172.21.15.37:6789/0 conn(0x7fc4cc163380 legacy=0x7fc4cc163810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4bc0039b0 0x7fc4bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4cc0cf990 0x7fc4bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4bc004d40 0x7fc4bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc0d0310 con 0x7fc4bc0039b0 2024-04-26T20:24:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc0b3760 con 0x7fc4bc004d40 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc4bc00b9c0 con 0x7fc4cc0cf990 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.207+0000 7fc4d0985640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4bc0039b0 0x7fc4bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.208+0000 7fc4cbfff640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4cc0cf990 0x7fc4bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.208+0000 7fc4d1186640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4bc004d40 0x7fc4bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.208+0000 7fc4d0985640 1 -- 172.21.15.37:0/2565709775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4cc0cf990 msgr2=0x7fc4bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.208+0000 7fc4d0985640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4cc0cf990 0x7fc4bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.208+0000 7fc4d0985640 1 -- 172.21.15.37:0/2565709775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4bc004d40 msgr2=0x7fc4bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.208+0000 7fc4d0985640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4bc004d40 0x7fc4bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.208+0000 7fc4d0985640 1 -- 172.21.15.37:0/2565709775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4bc003520 con 0x7fc4bc0039b0 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.208+0000 7fc4d0985640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4bc0039b0 0x7fc4bc003dd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009ac0 tx=0x7fc4bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24db928b837667ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.209+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0036070 con 0x7fc4bc0039b0 2024-04-26T20:24:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.209+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c0031400 con 0x7fc4bc0039b0 2024-04-26T20:24:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.209+0000 7fc4cb7fe640 1 -- 172.21.15.37:0/2565709775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0009690 con 0x7fc4bc0039b0 2024-04-26T20:24:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.209+0000 7fc4d2be9640 1 -- 172.21.15.37:0/2565709775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4bc0039b0 msgr2=0x7fc4bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.209+0000 7fc4d2be9640 1 --2- 172.21.15.37:0/2565709775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4bc0039b0 0x7fc4bc003dd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009ac0 tx=0x7fc4bc005830 comp rx=0 tx=0).stop 2024-04-26T20:24:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.209+0000 7fc4d1186640 1 -- 172.21.15.37:0/2565709775 reap_dead start 2024-04-26T20:24:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.209+0000 7fc4d2be9640 1 -- 172.21.15.37:0/2565709775 shutdown_connections 2024-04-26T20:24:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.209+0000 7fc4d2be9640 1 -- 172.21.15.37:0/2565709775 >> 172.21.15.37:0/2565709775 conn(0x7fc4cc0d2f80 msgr2=0x7fc4cc0d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.209+0000 7fc4d2be9640 1 -- 172.21.15.37:0/2565709775 shutdown_connections 2024-04-26T20:24:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.209+0000 7fc4d2be9640 1 -- 172.21.15.37:0/2565709775 wait complete. 2024-04-26T20:24:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.210+0000 7fc4d2be9640 1 Processor -- start 2024-04-26T20:24:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.210+0000 7fc4d2be9640 1 -- start start 2024-04-26T20:24:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.211+0000 7fc4d2be9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4bc0039b0 0x7fc4cc15fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.211+0000 7fc4d2be9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4cc1573d0 0x7fc4cc15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.211+0000 7fc4d2be9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4cc15c820 0x7fc4cc1ea750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.211+0000 7fc4d2be9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc06e590 con 0x7fc4cc15c820 2024-04-26T20:24:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.211+0000 7fc4d2be9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc06e2b0 con 0x7fc4cc1573d0 2024-04-26T20:24:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.211+0000 7fc4d2be9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc4cc06e870 con 0x7fc4bc0039b0 2024-04-26T20:24:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.211+0000 7fc4d1186640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4cc15c820 0x7fc4cc1ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.211+0000 7fc4d1186640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4cc15c820 0x7fc4cc1ea750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43670/0 (socket says 172.21.15.37:43670) 2024-04-26T20:24:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.211+0000 7fc4d1186640 1 -- 172.21.15.37:0/4146802188 learned_addr learned my addr 172.21.15.37:0/4146802188 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.211+0000 7fc4d0985640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4bc0039b0 0x7fc4cc15fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.211+0000 7fc4cbfff640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4cc1573d0 0x7fc4cc15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.212+0000 7fc4d1186640 1 -- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4bc0039b0 msgr2=0x7fc4cc15fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.212+0000 7fc4d1186640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4bc0039b0 0x7fc4cc15fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.212+0000 7fc4d1186640 1 -- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4cc1573d0 msgr2=0x7fc4cc15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.212+0000 7fc4d1186640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4cc1573d0 0x7fc4cc15c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.212+0000 7fc4d1186640 1 -- 172.21.15.37:0/4146802188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc0c7ba0 con 0x7fc4cc15c820 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.212+0000 7fc4d1186640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4cc15c820 0x7fc4cc1ea750 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0b9030 tx=0x7fc4c400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.212+0000 7fc4c97fa640 1 -- 172.21.15.37:0/4146802188 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4017020 con 0x7fc4cc15c820 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.212+0000 7fc4c97fa640 1 -- 172.21.15.37:0/4146802188 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c400ec10 con 0x7fc4cc15c820 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.212+0000 7fc4d2be9640 1 -- 172.21.15.37:0/4146802188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4cc0c7e80 con 0x7fc4cc15c820 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.212+0000 7fc4c97fa640 1 -- 172.21.15.37:0/4146802188 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4013420 con 0x7fc4cc15c820 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.212+0000 7fc4d2be9640 1 -- 172.21.15.37:0/4146802188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4cc0c8440 con 0x7fc4cc15c820 2024-04-26T20:24:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.213+0000 7fc4d2be9640 1 -- 172.21.15.37:0/4146802188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4cc06a360 con 0x7fc4cc15c820 2024-04-26T20:24:07.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.214+0000 7fc4c97fa640 1 -- 172.21.15.37:0/4146802188 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fc4c401a020 con 0x7fc4cc15c820 2024-04-26T20:24:07.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.214+0000 7fc4c97fa640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc4b006cbc0 0x7fc4b006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.214+0000 7fc4c97fa640 1 -- 172.21.15.37:0/4146802188 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7fc4c408bd50 con 0x7fc4cc15c820 2024-04-26T20:24:07.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.214+0000 7fc4d0985640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc4b006cbc0 0x7fc4b006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.215+0000 7fc4d0985640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc4b006cbc0 0x7fc4b006f080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0033000 tx=0x7fc4c0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:07.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.218+0000 7fc4c97fa640 1 -- 172.21.15.37:0/4146802188 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4c4021030 con 0x7fc4cc15c820 2024-04-26T20:24:07.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.406+0000 7fc4d2be9640 1 -- 172.21.15.37:0/4146802188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc4cc0aff70 con 0x7fc4cc15c820 2024-04-26T20:24:07.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.406+0000 7fc4c97fa640 1 -- 172.21.15.37:0/4146802188 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7fc4cc0b2550 con 0x7fc4cc15c820 2024-04-26T20:24:07.407 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:24:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 -- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc4b006cbc0 msgr2=0x7fc4b006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc4b006cbc0 0x7fc4b006f080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0033000 tx=0x7fc4c0008030 comp rx=0 tx=0).stop 2024-04-26T20:24:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 -- 172.21.15.37:0/4146802188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4cc15c820 msgr2=0x7fc4cc1ea750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4cc15c820 0x7fc4cc1ea750 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0b9030 tx=0x7fc4c400e530 comp rx=0 tx=0).stop 2024-04-26T20:24:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 -- 172.21.15.37:0/4146802188 shutdown_connections 2024-04-26T20:24:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4cc15c820 0x7fc4cc1ea750 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4cc1573d0 0x7fc4cc15c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc4b006cbc0 0x7fc4b006f080 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 --2- 172.21.15.37:0/4146802188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4bc0039b0 0x7fc4cc15fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 -- 172.21.15.37:0/4146802188 >> 172.21.15.37:0/4146802188 conn(0x7fc4cc0d2f80 msgr2=0x7fc4cc0b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 -- 172.21.15.37:0/4146802188 shutdown_connections 2024-04-26T20:24:07.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.408+0000 7fc4d2be9640 1 -- 172.21.15.37:0/4146802188 wait complete. 2024-04-26T20:24:07.421 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:24:07.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.521+0000 7f55621f9640 1 Processor -- start 2024-04-26T20:24:07.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.521+0000 7f55621f9640 1 -- start start 2024-04-26T20:24:07.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.522+0000 7f55621f9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f555c162e40 0x7f555c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.522+0000 7f55621f9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f555c0b2050 0x7f555c0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.522+0000 7f55621f9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555c0d0150 con 0x7f555c0cfd70 2024-04-26T20:24:07.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.522+0000 7f55621f9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f555c163220 con 0x7f555c0b2050 2024-04-26T20:24:07.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.522+0000 7f55621f9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f555c0d0eb0 con 0x7f555c162e40 2024-04-26T20:24:07.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.522+0000 7f555affd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f555c0b2050 0x7f555c0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.522+0000 7f555bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f555c0cfd70 0x7f555c0ce010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59700/0 (socket says 172.21.15.37:59700) 2024-04-26T20:24:07.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.522+0000 7f555bfff640 1 -- 172.21.15.37:0/1497010980 learned_addr learned my addr 172.21.15.37:0/1497010980 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:07.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.523+0000 7f555b7fe640 1 --2- 172.21.15.37:0/1497010980 >> v2:172.21.15.134:3300/0 conn(0x7f555c162e40 0x7f555c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.523+0000 7f555affd640 1 -- 172.21.15.37:0/1497010980 >> v2:172.21.15.134:3300/0 conn(0x7f555c162e40 msgr2=0x7f555c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:07.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.523+0000 7f555affd640 1 --2- 172.21.15.37:0/1497010980 >> v2:172.21.15.134:3300/0 conn(0x7f555c162e40 0x7f555c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.523+0000 7f555affd640 1 -- 172.21.15.37:0/1497010980 >> v1:172.21.15.37:6789/0 conn(0x7f555c0cfd70 legacy=0x7f555c0ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.523+0000 7f555a7fc640 1 -- 172.21.15.37:0/1497010980 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1857656749 0 0) 0x7f555c0d0150 con 0x7f555c0cfd70 2024-04-26T20:24:07.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.523+0000 7f555affd640 1 -- 172.21.15.37:0/1497010980 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c0b1d70 con 0x7f555c0b2050 2024-04-26T20:24:07.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.523+0000 7f555affd640 1 --2- 172.21.15.37:0/1497010980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f555c0b2050 0x7f555c0cf830 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f555c075e20 tx=0x7f555002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f8614e339806da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:07.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.524+0000 7f555a7fc640 1 -- 172.21.15.37:0/1497010980 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550037070 con 0x7f555c0b2050 2024-04-26T20:24:07.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.524+0000 7f555a7fc640 1 -- 172.21.15.37:0/1497010980 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5550032440 con 0x7f555c0b2050 2024-04-26T20:24:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.524+0000 7f555a7fc640 1 -- 172.21.15.37:0/1497010980 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555002ae60 con 0x7f555c0b2050 2024-04-26T20:24:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.524+0000 7f55621f9640 1 -- 172.21.15.37:0/1497010980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f555c0b2050 msgr2=0x7f555c0cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.524+0000 7f55621f9640 1 --2- 172.21.15.37:0/1497010980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f555c0b2050 0x7f555c0cf830 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f555c075e20 tx=0x7f555002a110 comp rx=0 tx=0).stop 2024-04-26T20:24:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.524+0000 7f55621f9640 1 -- 172.21.15.37:0/1497010980 shutdown_connections 2024-04-26T20:24:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.524+0000 7f55621f9640 1 --2- 172.21.15.37:0/1497010980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f555c0b2050 0x7f555c0cf830 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.524+0000 7f55621f9640 1 --2- 172.21.15.37:0/1497010980 >> v2:172.21.15.134:3300/0 conn(0x7f555c162e40 0x7f555c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.524+0000 7f55621f9640 1 -- 172.21.15.37:0/1497010980 >> 172.21.15.37:0/1497010980 conn(0x7f555c09ba50 msgr2=0x7f555c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.524+0000 7f55621f9640 1 -- 172.21.15.37:0/1497010980 shutdown_connections 2024-04-26T20:24:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.524+0000 7f55621f9640 1 -- 172.21.15.37:0/1497010980 wait complete. 2024-04-26T20:24:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.525+0000 7f55621f9640 1 Processor -- start 2024-04-26T20:24:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.526+0000 7f55621f9640 1 -- start start 2024-04-26T20:24:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.526+0000 7f55621f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f555c0b2050 0x7f555c0b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.526+0000 7f55621f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f555c0cfd70 0x7f555c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.526+0000 7f55621f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f555c162e40 0x7f555c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.526+0000 7f55621f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f555c0d1f20 con 0x7f555c162e40 2024-04-26T20:24:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.526+0000 7f55621f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f555c0d1c40 con 0x7f555c0b2050 2024-04-26T20:24:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.526+0000 7f55621f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f555c0b24e0 con 0x7f555c0cfd70 2024-04-26T20:24:07.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.526+0000 7f555bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f555c162e40 0x7f555c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f555bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f555c162e40 0x7f555c0b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43698/0 (socket says 172.21.15.37:43698) 2024-04-26T20:24:07.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f555bfff640 1 -- 172.21.15.37:0/2874987787 learned_addr learned my addr 172.21.15.37:0/2874987787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:07.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f555b7fe640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f555c0b2050 0x7f555c0b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f555affd640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f555c0cfd70 0x7f555c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f555bfff640 1 -- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f555c0cfd70 msgr2=0x7f555c15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f555bfff640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f555c0cfd70 0x7f555c15a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f555bfff640 1 -- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f555c0b2050 msgr2=0x7f555c0b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f555bfff640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f555c0b2050 0x7f555c0b7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f555bfff640 1 -- 172.21.15.37:0/2874987787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c0b85b0 con 0x7f555c162e40 2024-04-26T20:24:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f555bfff640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f555c162e40 0x7f555c0b4fd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f555400caf0 tx=0x7f555400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f5543fff640 1 -- 172.21.15.37:0/2874987787 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555400aca0 con 0x7f555c162e40 2024-04-26T20:24:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f5543fff640 1 -- 172.21.15.37:0/2874987787 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555400ae40 con 0x7f555c162e40 2024-04-26T20:24:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f55621f9640 1 -- 172.21.15.37:0/2874987787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555c0b82d0 con 0x7f555c162e40 2024-04-26T20:24:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f5543fff640 1 -- 172.21.15.37:0/2874987787 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554012640 con 0x7f555c162e40 2024-04-26T20:24:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.527+0000 7f55621f9640 1 -- 172.21.15.37:0/2874987787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f555c0b2b90 con 0x7f555c162e40 2024-04-26T20:24:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.528+0000 7f55621f9640 1 -- 172.21.15.37:0/2874987787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f555c06a360 con 0x7f555c162e40 2024-04-26T20:24:07.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.529+0000 7f5543fff640 1 -- 172.21.15.37:0/2874987787 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f55540128a0 con 0x7f555c162e40 2024-04-26T20:24:07.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.529+0000 7f5543fff640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f554806cbc0 0x7f554806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.529+0000 7f5543fff640 1 -- 172.21.15.37:0/2874987787 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f555408c190 con 0x7f555c162e40 2024-04-26T20:24:07.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.529+0000 7f555b7fe640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f554806cbc0 0x7f554806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.530+0000 7f555b7fe640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f554806cbc0 0x7f554806f080 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f554c0079a0 tx=0x7f554c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:07.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.534+0000 7f5543fff640 1 -- 172.21.15.37:0/2874987787 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f555405a000 con 0x7f555c162e40 2024-04-26T20:24:07.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.724+0000 7f55621f9640 1 -- 172.21.15.37:0/2874987787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f555c076100 con 0x7f555c162e40 2024-04-26T20:24:07.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.724+0000 7f5543fff640 1 -- 172.21.15.37:0/2874987787 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) ==== 84+0+65 (secure 0 0 0) 0x7f555405e300 con 0x7f555c162e40 2024-04-26T20:24:07.726 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:07.726 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-26T20:24:07.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:07.726 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-26T20:24:07.726 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-26T20:24:07.726 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:07.726 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.726+0000 7f55621f9640 1 -- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f554806cbc0 msgr2=0x7f554806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.726+0000 7f55621f9640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f554806cbc0 0x7f554806f080 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f554c0079a0 tx=0x7f554c008040 comp rx=0 tx=0).stop 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.726+0000 7f55621f9640 1 -- 172.21.15.37:0/2874987787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f555c162e40 msgr2=0x7f555c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.726+0000 7f55621f9640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f555c162e40 0x7f555c0b4fd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f555400caf0 tx=0x7f555400a060 comp rx=0 tx=0).stop 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.726+0000 7f55621f9640 1 -- 172.21.15.37:0/2874987787 shutdown_connections 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.726+0000 7f55621f9640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f555c162e40 0x7f555c0b4fd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.726+0000 7f55621f9640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f554806cbc0 0x7f554806f080 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.726+0000 7f55621f9640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f555c0cfd70 0x7f555c15a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.726+0000 7f55621f9640 1 --2- 172.21.15.37:0/2874987787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f555c0b2050 0x7f555c0b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.726+0000 7f55621f9640 1 -- 172.21.15.37:0/2874987787 >> 172.21.15.37:0/2874987787 conn(0x7f555c09ba50 msgr2=0x7f555c0ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.727+0000 7f55621f9640 1 -- 172.21.15.37:0/2874987787 shutdown_connections 2024-04-26T20:24:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.727+0000 7f55621f9640 1 -- 172.21.15.37:0/2874987787 wait complete. 2024-04-26T20:24:07.739 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:24:07.739 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:24:07.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.842+0000 7f8d0750a640 1 Processor -- start 2024-04-26T20:24:07.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.843+0000 7f8d0750a640 1 -- start start 2024-04-26T20:24:07.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.843+0000 7f8d0750a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8d000683f0 0x7f8d000ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.843+0000 7f8d0750a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8d0006d590 0x7f8d000b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.843+0000 7f8d0750a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d00070720 con 0x7f8d0015f350 2024-04-26T20:24:07.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.843+0000 7f8d0750a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f8d0015f730 con 0x7f8d000683f0 2024-04-26T20:24:07.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.843+0000 7f8d0750a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f8d00163de0 con 0x7f8d0006d590 2024-04-26T20:24:07.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.843+0000 7f8d04aa5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8d000683f0 0x7f8d000ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.843+0000 7f8d04aa5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8d000683f0 0x7f8d000ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43708/0 (socket says 172.21.15.37:43708) 2024-04-26T20:24:07.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.844+0000 7f8d04aa5640 1 -- 172.21.15.37:0/3343267816 learned_addr learned my addr 172.21.15.37:0/3343267816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:07.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.844+0000 7f8d05aa7640 1 --2- 172.21.15.37:0/3343267816 >> v2:172.21.15.134:3300/0 conn(0x7f8d0006d590 0x7f8d000b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.844+0000 7f8d04aa5640 1 -- 172.21.15.37:0/3343267816 >> v2:172.21.15.134:3300/0 conn(0x7f8d0006d590 msgr2=0x7f8d000b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:07.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.844+0000 7f8d04aa5640 1 --2- 172.21.15.37:0/3343267816 >> v2:172.21.15.134:3300/0 conn(0x7f8d0006d590 0x7f8d000b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.844+0000 7f8d04aa5640 1 -- 172.21.15.37:0/3343267816 >> v1:172.21.15.134:6789/0 conn(0x7f8d0015f350 legacy=0x7f8d0006ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:07.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.844+0000 7f8d04aa5640 1 -- 172.21.15.37:0/3343267816 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0006c3e0 con 0x7f8d000683f0 2024-04-26T20:24:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.844+0000 7f8d04aa5640 1 --2- 172.21.15.37:0/3343267816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d000683f0 0x7f8d000ae8c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8d0006ec90 tx=0x7f8cf402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64731f45724949df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.845+0000 7f8cfffff640 1 -- 172.21.15.37:0/3343267816 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4037070 con 0x7f8d000683f0 2024-04-26T20:24:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.845+0000 7f8cfffff640 1 -- 172.21.15.37:0/3343267816 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf4032440 con 0x7f8d000683f0 2024-04-26T20:24:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.845+0000 7f8cfffff640 1 -- 172.21.15.37:0/3343267816 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4032810 con 0x7f8d000683f0 2024-04-26T20:24:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.845+0000 7f8d0750a640 1 -- 172.21.15.37:0/3343267816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d000683f0 msgr2=0x7f8d000ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.845+0000 7f8d0750a640 1 --2- 172.21.15.37:0/3343267816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d000683f0 0x7f8d000ae8c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8d0006ec90 tx=0x7f8cf402a230 comp rx=0 tx=0).stop 2024-04-26T20:24:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.845+0000 7f8d0750a640 1 -- 172.21.15.37:0/3343267816 shutdown_connections 2024-04-26T20:24:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.845+0000 7f8d0750a640 1 --2- 172.21.15.37:0/3343267816 >> v2:172.21.15.134:3300/0 conn(0x7f8d0006d590 0x7f8d000b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.845+0000 7f8d0750a640 1 --2- 172.21.15.37:0/3343267816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d000683f0 0x7f8d000ae8c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.845+0000 7f8d0750a640 1 -- 172.21.15.37:0/3343267816 >> 172.21.15.37:0/3343267816 conn(0x7f8d000d15c0 msgr2=0x7f8d000d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.845+0000 7f8d0750a640 1 -- 172.21.15.37:0/3343267816 shutdown_connections 2024-04-26T20:24:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.845+0000 7f8d0750a640 1 -- 172.21.15.37:0/3343267816 wait complete. 2024-04-26T20:24:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.846+0000 7f8d0750a640 1 Processor -- start 2024-04-26T20:24:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.846+0000 7f8d0750a640 1 -- start start 2024-04-26T20:24:07.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.847+0000 7f8d0750a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d000683f0 0x7f8d00154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.847+0000 7f8d0750a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d0006d590 0x7f8d00159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.847+0000 7f8d0750a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d0015f350 0x7f8d00157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.847+0000 7f8d0750a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00163b00 con 0x7f8d0015f350 2024-04-26T20:24:07.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.847+0000 7f8d0750a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0006c9f0 con 0x7f8d0006d590 2024-04-26T20:24:07.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.847+0000 7f8d0750a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0006c680 con 0x7f8d000683f0 2024-04-26T20:24:07.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.847+0000 7f8d05aa7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d0015f350 0x7f8d00157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.847+0000 7f8d05aa7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d0015f350 0x7f8d00157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43724/0 (socket says 172.21.15.37:43724) 2024-04-26T20:24:07.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.847+0000 7f8d05aa7640 1 -- 172.21.15.37:0/2563670294 learned_addr learned my addr 172.21.15.37:0/2563670294 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.847+0000 7f8d052a6640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d000683f0 0x7f8d00154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.847+0000 7f8d04aa5640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d0006d590 0x7f8d00159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.848+0000 7f8d05aa7640 1 -- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d000683f0 msgr2=0x7f8d00154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.848+0000 7f8d05aa7640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d000683f0 0x7f8d00154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.848+0000 7f8d05aa7640 1 -- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d0006d590 msgr2=0x7f8d00159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.848+0000 7f8d05aa7640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d0006d590 0x7f8d00159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.848+0000 7f8d05aa7640 1 -- 172.21.15.37:0/2563670294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d00158da0 con 0x7f8d0015f350 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.848+0000 7f8d05aa7640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d0015f350 0x7f8d00157c70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0007e40 tx=0x7f8cf000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.848+0000 7f8cfdffb640 1 -- 172.21.15.37:0/2563670294 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0014620 con 0x7f8d0015f350 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.848+0000 7f8cfdffb640 1 -- 172.21.15.37:0/2563670294 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf0019070 con 0x7f8d0015f350 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.848+0000 7f8d0750a640 1 -- 172.21.15.37:0/2563670294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d00155460 con 0x7f8d0015f350 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.848+0000 7f8cfdffb640 1 -- 172.21.15.37:0/2563670294 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0014a20 con 0x7f8d0015f350 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.848+0000 7f8d0750a640 1 -- 172.21.15.37:0/2563670294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d00159340 con 0x7f8d0015f350 2024-04-26T20:24:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.849+0000 7f8d0750a640 1 -- 172.21.15.37:0/2563670294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0006a360 con 0x7f8d0015f350 2024-04-26T20:24:07.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.850+0000 7f8cfdffb640 1 -- 172.21.15.37:0/2563670294 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f8cf0014bc0 con 0x7f8d0015f350 2024-04-26T20:24:07.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.850+0000 7f8cfdffb640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8cec06cbc0 0x7f8cec06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.850+0000 7f8cfdffb640 1 -- 172.21.15.37:0/2563670294 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f8cf008c3b0 con 0x7f8d0015f350 2024-04-26T20:24:07.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.850+0000 7f8d052a6640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8cec06cbc0 0x7f8cec06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.851+0000 7f8d052a6640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8cec06cbc0 0x7f8cec06f080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8006fd0 tx=0x7f8cf8009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:07.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:07.854+0000 7f8cfdffb640 1 -- 172.21.15.37:0/2563670294 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8cf005a1d0 con 0x7f8d0015f350 2024-04-26T20:24:08.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.042+0000 7f8d0750a640 1 -- 172.21.15.37:0/2563670294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8d000681d0 con 0x7f8d0015f350 2024-04-26T20:24:08.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.381+0000 7f8cfdffb640 1 -- 172.21.15.37:0/2563670294 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) ==== 138+0+0 (secure 0 0 0) 0x7f8cf005e4d0 con 0x7f8d0015f350 2024-04-26T20:24:08.382 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-26T20:24:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 -- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8cec06cbc0 msgr2=0x7f8cec06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8cec06cbc0 0x7f8cec06f080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8006fd0 tx=0x7f8cf8009060 comp rx=0 tx=0).stop 2024-04-26T20:24:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 -- 172.21.15.37:0/2563670294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d0015f350 msgr2=0x7f8d00157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d0015f350 0x7f8d00157c70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0007e40 tx=0x7f8cf000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 -- 172.21.15.37:0/2563670294 shutdown_connections 2024-04-26T20:24:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d0015f350 0x7f8d00157c70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d0006d590 0x7f8d00159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8cec06cbc0 0x7f8cec06f080 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 --2- 172.21.15.37:0/2563670294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d000683f0 0x7f8d00154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 -- 172.21.15.37:0/2563670294 >> 172.21.15.37:0/2563670294 conn(0x7f8d000d15c0 msgr2=0x7f8d0015d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 -- 172.21.15.37:0/2563670294 shutdown_connections 2024-04-26T20:24:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.383+0000 7f8d0750a640 1 -- 172.21.15.37:0/2563670294 wait complete. 2024-04-26T20:24:08.399 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:24:08.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.508+0000 7f351c647640 1 Processor -- start 2024-04-26T20:24:08.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.508+0000 7f351c647640 1 -- start start 2024-04-26T20:24:08.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351c647640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f35140d1c30 0x7f35140b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:08.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351c647640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35140d2010 0x7f35140d5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:08.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351c647640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35140b63e0 0x7f35140d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:08.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351c647640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f35140d4380 con 0x7f35140b63e0 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351c647640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f35140d3b20 con 0x7f35140d1c30 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351c647640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f35140d3dc0 con 0x7f35140d2010 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351abe4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35140b63e0 0x7f35140d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351abe4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35140b63e0 0x7f35140d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43738/0 (socket says 172.21.15.37:43738) 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351abe4640 1 -- 172.21.15.37:0/3507195017 learned_addr learned my addr 172.21.15.37:0/3507195017 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f3519be2640 1 --2- 172.21.15.37:0/3507195017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35140d2010 0x7f35140d5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351a3e3640 1 --2- 172.21.15.37:0/3507195017 >> v2:172.21.15.134:3300/0 conn(0x7f35140d1c30 0x7f35140b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351abe4640 1 -- 172.21.15.37:0/3507195017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35140d2010 msgr2=0x7f35140d5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351abe4640 1 --2- 172.21.15.37:0/3507195017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35140d2010 0x7f35140d5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351abe4640 1 -- 172.21.15.37:0/3507195017 >> v2:172.21.15.134:3300/0 conn(0x7f35140d1c30 msgr2=0x7f35140b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351abe4640 1 --2- 172.21.15.37:0/3507195017 >> v2:172.21.15.134:3300/0 conn(0x7f35140d1c30 0x7f35140b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351abe4640 1 -- 172.21.15.37:0/3507195017 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35140d40a0 con 0x7f35140b63e0 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f351a3e3640 1 --2- 172.21.15.37:0/3507195017 >> v2:172.21.15.134:3300/0 conn(0x7f35140d1c30 0x7f35140b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:08.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.509+0000 7f3519be2640 1 --2- 172.21.15.37:0/3507195017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35140d2010 0x7f35140d5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:08.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.510+0000 7f351abe4640 1 --2- 172.21.15.37:0/3507195017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35140b63e0 0x7f35140d0710 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f351408d820 tx=0x7f350c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf3f6acdfc338c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:08.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.510+0000 7f35193e1640 1 -- 172.21.15.37:0/3507195017 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c036070 con 0x7f35140b63e0 2024-04-26T20:24:08.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.510+0000 7f35193e1640 1 -- 172.21.15.37:0/3507195017 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f350c031600 con 0x7f35140b63e0 2024-04-26T20:24:08.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.510+0000 7f35193e1640 1 -- 172.21.15.37:0/3507195017 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c0319d0 con 0x7f35140b63e0 2024-04-26T20:24:08.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.511+0000 7f351c647640 1 -- 172.21.15.37:0/3507195017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35140b63e0 msgr2=0x7f35140d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:08.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.511+0000 7f351c647640 1 --2- 172.21.15.37:0/3507195017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35140b63e0 0x7f35140d0710 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f351408d820 tx=0x7f350c02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.511+0000 7f351c647640 1 -- 172.21.15.37:0/3507195017 shutdown_connections 2024-04-26T20:24:08.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.511+0000 7f351c647640 1 --2- 172.21.15.37:0/3507195017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35140b63e0 0x7f35140d0710 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.511+0000 7f351c647640 1 --2- 172.21.15.37:0/3507195017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35140d2010 0x7f35140d5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.511+0000 7f351c647640 1 --2- 172.21.15.37:0/3507195017 >> v2:172.21.15.134:3300/0 conn(0x7f35140d1c30 0x7f35140b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.511+0000 7f351c647640 1 -- 172.21.15.37:0/3507195017 >> 172.21.15.37:0/3507195017 conn(0x7f351409e560 msgr2=0x7f35140d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:08.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.511+0000 7f351c647640 1 -- 172.21.15.37:0/3507195017 shutdown_connections 2024-04-26T20:24:08.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.511+0000 7f351c647640 1 -- 172.21.15.37:0/3507195017 wait complete. 2024-04-26T20:24:08.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.512+0000 7f351c647640 1 Processor -- start 2024-04-26T20:24:08.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.512+0000 7f351c647640 1 -- start start 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351c647640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35140b63e0 0x7f35140cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351c647640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35140d1c30 0x7f35141f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351c647640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35140d2010 0x7f35140cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351c647640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f35140cfea0 con 0x7f35140b63e0 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351c647640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35140b8620 con 0x7f35140d2010 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351c647640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f35140d60f0 con 0x7f35140d1c30 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351a3e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35140b63e0 0x7f35140cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351a3e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35140b63e0 0x7f35140cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43744/0 (socket says 172.21.15.37:43744) 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351a3e3640 1 -- 172.21.15.37:0/3289650283 learned_addr learned my addr 172.21.15.37:0/3289650283 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351a3e3640 1 -- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35140d1c30 msgr2=0x7f35141f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351a3e3640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35140d1c30 0x7f35141f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351a3e3640 1 -- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35140d2010 msgr2=0x7f35140cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:08.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351a3e3640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35140d2010 0x7f35140cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351a3e3640 1 -- 172.21.15.37:0/3289650283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35140ce290 con 0x7f35140b63e0 2024-04-26T20:24:08.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351a3e3640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35140b63e0 0x7f35140cc360 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f351000c7f0 tx=0x7f351000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:08.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f350affd640 1 -- 172.21.15.37:0/3289650283 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510015070 con 0x7f35140b63e0 2024-04-26T20:24:08.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f350affd640 1 -- 172.21.15.37:0/3289650283 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3510007820 con 0x7f35140b63e0 2024-04-26T20:24:08.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351c647640 1 -- 172.21.15.37:0/3289650283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35140cc960 con 0x7f35140b63e0 2024-04-26T20:24:08.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f350affd640 1 -- 172.21.15.37:0/3289650283 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510007c10 con 0x7f35140b63e0 2024-04-26T20:24:08.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.513+0000 7f351c647640 1 -- 172.21.15.37:0/3289650283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35140ccf20 con 0x7f35140b63e0 2024-04-26T20:24:08.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.514+0000 7f351c647640 1 -- 172.21.15.37:0/3289650283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351406a360 con 0x7f35140b63e0 2024-04-26T20:24:08.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.515+0000 7f350affd640 1 -- 172.21.15.37:0/3289650283 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f3510018070 con 0x7f35140b63e0 2024-04-26T20:24:08.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.515+0000 7f350affd640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34f806cbc0 0x7f34f806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:08.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.515+0000 7f350affd640 1 -- 172.21.15.37:0/3289650283 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5227+0+0 (secure 0 0 0) 0x7f351008c020 con 0x7f35140b63e0 2024-04-26T20:24:08.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.516+0000 7f3519be2640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34f806cbc0 0x7f34f806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:08.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.516+0000 7f3519be2640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34f806cbc0 0x7f34f806f080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3504006fd0 tx=0x7f3504008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:08.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.520+0000 7f350affd640 1 -- 172.21.15.37:0/3289650283 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f351005a150 con 0x7f35140b63e0 2024-04-26T20:24:08.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:08.704+0000 7f351c647640 1 -- 172.21.15.37:0/3289650283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f35140d2980 con 0x7f35140b63e0 2024-04-26T20:24:09.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.388+0000 7f350affd640 1 -- 172.21.15.37:0/3289650283 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) ==== 170+0+0 (secure 0 0 0) 0x7f3510059a50 con 0x7f35140b63e0 2024-04-26T20:24:09.389 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-26T20:24:09.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.391+0000 7f351c647640 1 -- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34f806cbc0 msgr2=0x7f34f806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:09.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.391+0000 7f351c647640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34f806cbc0 0x7f34f806f080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3504006fd0 tx=0x7f3504008040 comp rx=0 tx=0).stop 2024-04-26T20:24:09.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.391+0000 7f351c647640 1 -- 172.21.15.37:0/3289650283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35140b63e0 msgr2=0x7f35140cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:09.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.391+0000 7f351c647640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35140b63e0 0x7f35140cc360 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f351000c7f0 tx=0x7f351000ccb0 comp rx=0 tx=0).stop 2024-04-26T20:24:09.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.393+0000 7f351c647640 1 -- 172.21.15.37:0/3289650283 shutdown_connections 2024-04-26T20:24:09.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.393+0000 7f351c647640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35140d2010 0x7f35140cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:09.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.393+0000 7f351c647640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34f806cbc0 0x7f34f806f080 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:09.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.393+0000 7f351c647640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35140d1c30 0x7f35141f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:09.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.393+0000 7f351c647640 1 --2- 172.21.15.37:0/3289650283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35140b63e0 0x7f35140cc360 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:09.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.393+0000 7f351c647640 1 -- 172.21.15.37:0/3289650283 >> 172.21.15.37:0/3289650283 conn(0x7f351409e560 msgr2=0x7f35140bab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:09.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.394+0000 7f351c647640 1 -- 172.21.15.37:0/3289650283 shutdown_connections 2024-04-26T20:24:09.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.394+0000 7f351c647640 1 -- 172.21.15.37:0/3289650283 wait complete. 2024-04-26T20:24:09.422 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:24:09.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.530+0000 7fbf06a99640 1 Processor -- start 2024-04-26T20:24:09.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.530+0000 7fbf06a99640 1 -- start start 2024-04-26T20:24:09.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbf06a99640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbf0015f9c0 0x7fbf000d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:09.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbf06a99640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbf000cf990 0x7fbf000cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:09.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbf06a99640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf000b2550 con 0x7fbf00163380 2024-04-26T20:24:09.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbf06a99640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fbf000d0340 con 0x7fbf000cf990 2024-04-26T20:24:09.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbf06a99640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fbf000b3760 con 0x7fbf0015f9c0 2024-04-26T20:24:09.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbefffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbf000cf990 0x7fbf000cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbf05036640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbf00163380 0x7fbf001637f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59708/0 (socket says 172.21.15.37:59708) 2024-04-26T20:24:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbf05036640 1 -- 172.21.15.37:0/738893157 learned_addr learned my addr 172.21.15.37:0/738893157 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbf04835640 1 --2- 172.21.15.37:0/738893157 >> v2:172.21.15.134:3300/0 conn(0x7fbf0015f9c0 0x7fbf000d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbefffff640 1 -- 172.21.15.37:0/738893157 >> v2:172.21.15.134:3300/0 conn(0x7fbf0015f9c0 msgr2=0x7fbf000d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbefffff640 1 --2- 172.21.15.37:0/738893157 >> v2:172.21.15.134:3300/0 conn(0x7fbf0015f9c0 0x7fbf000d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbefffff640 1 -- 172.21.15.37:0/738893157 >> v1:172.21.15.37:6789/0 conn(0x7fbf00163380 legacy=0x7fbf001637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbeff7fe640 1 -- 172.21.15.37:0/738893157 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2049062777 0 0) 0x7fbef8001490 con 0x7fbf00163380 2024-04-26T20:24:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.531+0000 7fbefffff640 1 -- 172.21.15.37:0/738893157 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0006dcf0 con 0x7fbf000cf990 2024-04-26T20:24:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.532+0000 7fbefffff640 1 --2- 172.21.15.37:0/738893157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf000cf990 0x7fbf000cfe00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbf000d2ee0 tx=0x7fbeec02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8c6134a0560e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.532+0000 7fbeff7fe640 1 -- 172.21.15.37:0/738893157 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec037070 con 0x7fbf000cf990 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.532+0000 7fbeff7fe640 1 -- 172.21.15.37:0/738893157 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeec032440 con 0x7fbf000cf990 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.532+0000 7fbeff7fe640 1 -- 172.21.15.37:0/738893157 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec02ae60 con 0x7fbf000cf990 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.533+0000 7fbf06a99640 1 -- 172.21.15.37:0/738893157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf000cf990 msgr2=0x7fbf000cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.533+0000 7fbf06a99640 1 --2- 172.21.15.37:0/738893157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf000cf990 0x7fbf000cfe00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbf000d2ee0 tx=0x7fbeec02a110 comp rx=0 tx=0).stop 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.533+0000 7fbf06a99640 1 -- 172.21.15.37:0/738893157 shutdown_connections 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.533+0000 7fbf06a99640 1 --2- 172.21.15.37:0/738893157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf000cf990 0x7fbf000cfe00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.533+0000 7fbf06a99640 1 --2- 172.21.15.37:0/738893157 >> v2:172.21.15.134:3300/0 conn(0x7fbf0015f9c0 0x7fbf000d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.533+0000 7fbf06a99640 1 -- 172.21.15.37:0/738893157 >> 172.21.15.37:0/738893157 conn(0x7fbf000d2f80 msgr2=0x7fbf000d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.533+0000 7fbf06a99640 1 -- 172.21.15.37:0/738893157 shutdown_connections 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.533+0000 7fbf06a99640 1 -- 172.21.15.37:0/738893157 wait complete. 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.534+0000 7fbf06a99640 1 Processor -- start 2024-04-26T20:24:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.534+0000 7fbf06a99640 1 -- start start 2024-04-26T20:24:09.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbf06a99640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf000cf990 0x7fbf00161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:09.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbf06a99640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0015f9c0 0x7fbf001f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:09.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbf06a99640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf00163380 0x7fbf001f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:09.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbf06a99640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0006e590 con 0x7fbf0015f9c0 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbf06a99640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0006e2b0 con 0x7fbf00163380 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbf06a99640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbf0006e870 con 0x7fbf000cf990 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbefffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0015f9c0 0x7fbf001f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbefffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0015f9c0 0x7fbf001f5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43774/0 (socket says 172.21.15.37:43774) 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbefffff640 1 -- 172.21.15.37:0/2544045848 learned_addr learned my addr 172.21.15.37:0/2544045848 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbf04835640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf000cf990 0x7fbf00161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbf05036640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf00163380 0x7fbf001f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbefffff640 1 -- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf000cf990 msgr2=0x7fbf00161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbefffff640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf000cf990 0x7fbf00161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbefffff640 1 -- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf00163380 msgr2=0x7fbf001f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbefffff640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf00163380 0x7fbf001f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbefffff640 1 -- 172.21.15.37:0/2544045848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf00162250 con 0x7fbf0015f9c0 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbefffff640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0015f9c0 0x7fbf001f5500 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbeec000c00 tx=0x7fbeec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.535+0000 7fbefd7fa640 1 -- 172.21.15.37:0/2544045848 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec037050 con 0x7fbf0015f9c0 2024-04-26T20:24:09.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.536+0000 7fbefd7fa640 1 -- 172.21.15.37:0/2544045848 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeec038040 con 0x7fbf0015f9c0 2024-04-26T20:24:09.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.536+0000 7fbefd7fa640 1 -- 172.21.15.37:0/2544045848 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004470 con 0x7fbf0015f9c0 2024-04-26T20:24:09.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.536+0000 7fbf06a99640 1 -- 172.21.15.37:0/2544045848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf00162530 con 0x7fbf0015f9c0 2024-04-26T20:24:09.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.536+0000 7fbf06a99640 1 -- 172.21.15.37:0/2544045848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0009cf60 con 0x7fbf0015f9c0 2024-04-26T20:24:09.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.537+0000 7fbf06a99640 1 -- 172.21.15.37:0/2544045848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0015cd20 con 0x7fbf0015f9c0 2024-04-26T20:24:09.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.537+0000 7fbefd7fa640 1 -- 172.21.15.37:0/2544045848 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fbeec030900 con 0x7fbf0015f9c0 2024-04-26T20:24:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.538+0000 7fbefd7fa640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbee406cbc0 0x7fbee406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.538+0000 7fbefd7fa640 1 -- 172.21.15.37:0/2544045848 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5609+0+0 (secure 0 0 0) 0x7fbeec0ac970 con 0x7fbf0015f9c0 2024-04-26T20:24:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.538+0000 7fbf04835640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbee406cbc0 0x7fbee406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.538+0000 7fbf04835640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbee406cbc0 0x7fbee406f080 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbef4006fd0 tx=0x7fbef4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.543+0000 7fbefd7fa640 1 -- 172.21.15.37:0/2544045848 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf0015cd20 con 0x7fbf0015f9c0 2024-04-26T20:24:09.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:09.771+0000 7fbf06a99640 1 -- 172.21.15.37:0/2544045848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbf0006a360 con 0x7fbf0015f9c0 2024-04-26T20:24:10.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.389+0000 7fbefd7fa640 1 -- 172.21.15.37:0/2544045848 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) ==== 172+0+0 (secure 0 0 0) 0x7fbeec07a460 con 0x7fbf0015f9c0 2024-04-26T20:24:10.391 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 6 and data pool 7 2024-04-26T20:24:10.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.394+0000 7fbf06a99640 1 -- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbee406cbc0 msgr2=0x7fbee406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:10.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.394+0000 7fbf06a99640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbee406cbc0 0x7fbee406f080 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbef4006fd0 tx=0x7fbef4008040 comp rx=0 tx=0).stop 2024-04-26T20:24:10.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.394+0000 7fbf06a99640 1 -- 172.21.15.37:0/2544045848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0015f9c0 msgr2=0x7fbf001f5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:10.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.394+0000 7fbf06a99640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0015f9c0 0x7fbf001f5500 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbeec000c00 tx=0x7fbeec0046c0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.394+0000 7fbf06a99640 1 -- 172.21.15.37:0/2544045848 shutdown_connections 2024-04-26T20:24:10.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.394+0000 7fbf06a99640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf00163380 0x7fbf001f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.394+0000 7fbf06a99640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0015f9c0 0x7fbf001f5500 secure :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbeec000c00 tx=0x7fbeec0046c0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.394+0000 7fbf06a99640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbee406cbc0 0x7fbee406f080 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbef4006fd0 tx=0x7fbef4008040 comp rx=0 tx=0).stop 2024-04-26T20:24:10.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.394+0000 7fbf06a99640 1 --2- 172.21.15.37:0/2544045848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf000cf990 0x7fbf00161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.394+0000 7fbf06a99640 1 -- 172.21.15.37:0/2544045848 >> 172.21.15.37:0/2544045848 conn(0x7fbf000d2f80 msgr2=0x7fbf0015d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:10.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.395+0000 7fbf06a99640 1 -- 172.21.15.37:0/2544045848 shutdown_connections 2024-04-26T20:24:10.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.395+0000 7fbf06a99640 1 -- 172.21.15.37:0/2544045848 wait complete. 2024-04-26T20:24:10.428 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:24:10.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.532+0000 7f206db1b640 1 Processor -- start 2024-04-26T20:24:10.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.532+0000 7f206db1b640 1 -- start start 2024-04-26T20:24:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.533+0000 7f206db1b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2068160a00 0x7f20680b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.533+0000 7f206db1b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f20680cf9c0 0x7f20680cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.533+0000 7f206db1b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f20680d0340 0x7f20680d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.533+0000 7f206db1b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f20680cf740 con 0x7f20680d0340 2024-04-26T20:24:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.533+0000 7f206db1b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f20680cf080 con 0x7f20680cf9c0 2024-04-26T20:24:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.533+0000 7f206db1b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f20680ce7e0 con 0x7f2068160a00 2024-04-26T20:24:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.533+0000 7f2067fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f20680d0340 0x7f20680d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.533+0000 7f2067fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f20680d0340 0x7f20680d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43776/0 (socket says 172.21.15.37:43776) 2024-04-26T20:24:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.533+0000 7f2067fff640 1 -- 172.21.15.37:0/4106546938 learned_addr learned my addr 172.21.15.37:0/4106546938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.533+0000 7f20677fe640 1 --2- 172.21.15.37:0/4106546938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2068160a00 0x7f20680b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.533+0000 7f2066ffd640 1 --2- 172.21.15.37:0/4106546938 >> v2:172.21.15.134:3300/0 conn(0x7f20680cf9c0 0x7f20680cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.534+0000 7f2067fff640 1 -- 172.21.15.37:0/4106546938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2068160a00 msgr2=0x7f20680b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.534+0000 7f2067fff640 1 --2- 172.21.15.37:0/4106546938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2068160a00 0x7f20680b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.534+0000 7f2067fff640 1 -- 172.21.15.37:0/4106546938 >> v2:172.21.15.134:3300/0 conn(0x7f20680cf9c0 msgr2=0x7f20680cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.534+0000 7f2067fff640 1 --2- 172.21.15.37:0/4106546938 >> v2:172.21.15.134:3300/0 conn(0x7f20680cf9c0 0x7f20680cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.534+0000 7f2067fff640 1 -- 172.21.15.37:0/4106546938 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20680ceac0 con 0x7f20680d0340 2024-04-26T20:24:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.534+0000 7f2066ffd640 1 --2- 172.21.15.37:0/4106546938 >> v2:172.21.15.134:3300/0 conn(0x7f20680cf9c0 0x7f20680cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.534+0000 7f2067fff640 1 --2- 172.21.15.37:0/4106546938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20680d0340 0x7f20680d0eb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f20680738c0 tx=0x7f206002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb6fb0f05626c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.534+0000 7f20667fc640 1 -- 172.21.15.37:0/4106546938 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060036070 con 0x7f20680d0340 2024-04-26T20:24:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.534+0000 7f20667fc640 1 -- 172.21.15.37:0/4106546938 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2060031600 con 0x7f20680d0340 2024-04-26T20:24:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.534+0000 7f20667fc640 1 -- 172.21.15.37:0/4106546938 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20600319d0 con 0x7f20680d0340 2024-04-26T20:24:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.535+0000 7f206db1b640 1 -- 172.21.15.37:0/4106546938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20680d0340 msgr2=0x7f20680d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.535+0000 7f206db1b640 1 --2- 172.21.15.37:0/4106546938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20680d0340 0x7f20680d0eb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f20680738c0 tx=0x7f206002a730 comp rx=0 tx=0).stop 2024-04-26T20:24:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.535+0000 7f206db1b640 1 -- 172.21.15.37:0/4106546938 shutdown_connections 2024-04-26T20:24:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.535+0000 7f206db1b640 1 --2- 172.21.15.37:0/4106546938 >> v2:172.21.15.134:3300/0 conn(0x7f20680cf9c0 0x7f20680cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.535+0000 7f206db1b640 1 --2- 172.21.15.37:0/4106546938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20680d0340 0x7f20680d0eb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.535+0000 7f206db1b640 1 --2- 172.21.15.37:0/4106546938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2068160a00 0x7f20680b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.535+0000 7f206db1b640 1 -- 172.21.15.37:0/4106546938 >> 172.21.15.37:0/4106546938 conn(0x7f206809ba50 msgr2=0x7f206809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.535+0000 7f206db1b640 1 -- 172.21.15.37:0/4106546938 shutdown_connections 2024-04-26T20:24:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.535+0000 7f206db1b640 1 -- 172.21.15.37:0/4106546938 wait complete. 2024-04-26T20:24:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.536+0000 7f206db1b640 1 Processor -- start 2024-04-26T20:24:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.536+0000 7f206db1b640 1 -- start start 2024-04-26T20:24:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f206db1b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20680cf9c0 0x7f20680c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f206db1b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20680d0340 0x7f206815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f206db1b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2068160a00 0x7f20680ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f206db1b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2068160de0 con 0x7f20680cf9c0 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f206db1b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20680d0770 con 0x7f2068160a00 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f206db1b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f20680b2690 con 0x7f20680d0340 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f20677fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20680cf9c0 0x7f20680c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f20677fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20680cf9c0 0x7f20680c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43778/0 (socket says 172.21.15.37:43778) 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f20677fe640 1 -- 172.21.15.37:0/1375385792 learned_addr learned my addr 172.21.15.37:0/1375385792 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f2066ffd640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20680d0340 0x7f206815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f20677fe640 1 -- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20680d0340 msgr2=0x7f206815a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f2067fff640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2068160a00 0x7f20680ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f20677fe640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20680d0340 0x7f206815a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f20677fe640 1 -- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2068160a00 msgr2=0x7f20680ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.537+0000 7f20677fe640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2068160a00 0x7f20680ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.538+0000 7f20677fe640 1 -- 172.21.15.37:0/1375385792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2068157910 con 0x7f20680cf9c0 2024-04-26T20:24:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.538+0000 7f20677fe640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20680cf9c0 0x7f20680c86b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f205800c7f0 tx=0x7f205800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.538+0000 7f204ffff640 1 -- 172.21.15.37:0/1375385792 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058015070 con 0x7f20680cf9c0 2024-04-26T20:24:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.538+0000 7f204ffff640 1 -- 172.21.15.37:0/1375385792 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2058004240 con 0x7f20680cf9c0 2024-04-26T20:24:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.538+0000 7f204ffff640 1 -- 172.21.15.37:0/1375385792 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058007620 con 0x7f20680cf9c0 2024-04-26T20:24:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.538+0000 7f206db1b640 1 -- 172.21.15.37:0/1375385792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20680cabf0 con 0x7f20680cf9c0 2024-04-26T20:24:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.538+0000 7f206db1b640 1 -- 172.21.15.37:0/1375385792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20680c7eb0 con 0x7f20680cf9c0 2024-04-26T20:24:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.539+0000 7f206db1b640 1 -- 172.21.15.37:0/1375385792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f206806a360 con 0x7f20680cf9c0 2024-04-26T20:24:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.540+0000 7f204ffff640 1 -- 172.21.15.37:0/1375385792 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f20580079d0 con 0x7f20680cf9c0 2024-04-26T20:24:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.540+0000 7f204ffff640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f205406cbc0 0x7f205406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.540+0000 7f204ffff640 1 -- 172.21.15.37:0/1375385792 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f205808c560 con 0x7f20680cf9c0 2024-04-26T20:24:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.540+0000 7f2066ffd640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f205406cbc0 0x7f205406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.541+0000 7f2066ffd640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f205406cbc0 0x7f205406f080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f20681f8800 tx=0x7f205c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:10.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.545+0000 7f204ffff640 1 -- 172.21.15.37:0/1375385792 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f205805a530 con 0x7f20680cf9c0 2024-04-26T20:24:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.731+0000 7f206db1b640 1 -- 172.21.15.37:0/1375385792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2068071340 con 0x7f20680cf9c0 2024-04-26T20:24:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.733+0000 7f204ffff640 1 -- 172.21.15.37:0/1375385792 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16638 (secure 0 0 0) 0x7f2058059e30 con 0x7f20680cf9c0 2024-04-26T20:24:10.734 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":45,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:24:09.772733+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:24:08.047629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:24:08.708738+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:24:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 -- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f205406cbc0 msgr2=0x7f205406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f205406cbc0 0x7f205406f080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f20681f8800 tx=0x7f205c008040 comp rx=0 tx=0).stop 2024-04-26T20:24:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 -- 172.21.15.37:0/1375385792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20680cf9c0 msgr2=0x7f20680c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20680cf9c0 0x7f20680c86b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f205800c7f0 tx=0x7f205800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 -- 172.21.15.37:0/1375385792 shutdown_connections 2024-04-26T20:24:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2068160a00 0x7f20680ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f205406cbc0 0x7f205406f080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20680d0340 0x7f206815a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 --2- 172.21.15.37:0/1375385792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20680cf9c0 0x7f20680c86b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 -- 172.21.15.37:0/1375385792 >> 172.21.15.37:0/1375385792 conn(0x7f206809ba50 msgr2=0x7f20680702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:10.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 -- 172.21.15.37:0/1375385792 shutdown_connections 2024-04-26T20:24:10.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.735+0000 7f206db1b640 1 -- 172.21.15.37:0/1375385792 wait complete. 2024-04-26T20:24:10.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:24:10.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.846+0000 7f1ca29e6640 1 Processor -- start 2024-04-26T20:24:10.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.847+0000 7f1ca29e6640 1 -- start start 2024-04-26T20:24:10.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.847+0000 7f1ca29e6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c9c15f380 0x7f1c9c0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.847+0000 7f1ca29e6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1c9c15f760 0x7f1c9c0d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.847+0000 7f1ca29e6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1c9c0d5b50 0x7f1c9c0cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.847+0000 7f1ca29e6640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9c0d43a0 con 0x7f1c9c0d5b50 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.847+0000 7f1ca29e6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9c0d3de0 con 0x7f1c9c15f760 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.847+0000 7f1ca29e6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9c0d40c0 con 0x7f1c9c15f380 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.848+0000 7f1ca0f83640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1c9c0d5b50 0x7f1c9c0cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.848+0000 7f1ca0f83640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1c9c0d5b50 0x7f1c9c0cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43782/0 (socket says 172.21.15.37:43782) 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.848+0000 7f1ca0f83640 1 -- 172.21.15.37:0/4211925269 learned_addr learned my addr 172.21.15.37:0/4211925269 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.848+0000 7f1c9b7fe640 1 --2- 172.21.15.37:0/4211925269 >> v2:172.21.15.134:3300/0 conn(0x7f1c9c15f760 0x7f1c9c0d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.848+0000 7f1c9bfff640 1 --2- 172.21.15.37:0/4211925269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c9c15f380 0x7f1c9c0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.848+0000 7f1ca0f83640 1 -- 172.21.15.37:0/4211925269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c9c15f380 msgr2=0x7f1c9c0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:10.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.848+0000 7f1ca0f83640 1 --2- 172.21.15.37:0/4211925269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c9c15f380 0x7f1c9c0d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.848+0000 7f1ca0f83640 1 -- 172.21.15.37:0/4211925269 >> v2:172.21.15.134:3300/0 conn(0x7f1c9c15f760 msgr2=0x7f1c9c0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:10.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.848+0000 7f1ca0f83640 1 --2- 172.21.15.37:0/4211925269 >> v2:172.21.15.134:3300/0 conn(0x7f1c9c15f760 0x7f1c9c0d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.848+0000 7f1ca0f83640 1 -- 172.21.15.37:0/4211925269 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c0b8280 con 0x7f1c9c0d5b50 2024-04-26T20:24:10.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.848+0000 7f1c9b7fe640 1 --2- 172.21.15.37:0/4211925269 >> v2:172.21.15.134:3300/0 conn(0x7f1c9c15f760 0x7f1c9c0d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:10.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1ca0f83640 1 --2- 172.21.15.37:0/4211925269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c9c0d5b50 0x7f1c9c0cfea0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c08d890 tx=0x7f1c9402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96c2a4872eb2470 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:10.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1c9affd640 1 -- 172.21.15.37:0/4211925269 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94036070 con 0x7f1c9c0d5b50 2024-04-26T20:24:10.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1c9affd640 1 -- 172.21.15.37:0/4211925269 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c94031600 con 0x7f1c9c0d5b50 2024-04-26T20:24:10.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1c9affd640 1 -- 172.21.15.37:0/4211925269 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c940319d0 con 0x7f1c9c0d5b50 2024-04-26T20:24:10.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1ca29e6640 1 -- 172.21.15.37:0/4211925269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c9c0d5b50 msgr2=0x7f1c9c0cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1ca29e6640 1 --2- 172.21.15.37:0/4211925269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c9c0d5b50 0x7f1c9c0cfea0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c08d890 tx=0x7f1c9402a730 comp rx=0 tx=0).stop 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1ca29e6640 1 -- 172.21.15.37:0/4211925269 shutdown_connections 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1ca29e6640 1 --2- 172.21.15.37:0/4211925269 >> v2:172.21.15.134:3300/0 conn(0x7f1c9c15f760 0x7f1c9c0d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1ca29e6640 1 --2- 172.21.15.37:0/4211925269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c9c0d5b50 0x7f1c9c0cfea0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1ca29e6640 1 --2- 172.21.15.37:0/4211925269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c9c15f380 0x7f1c9c0d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1ca29e6640 1 -- 172.21.15.37:0/4211925269 >> 172.21.15.37:0/4211925269 conn(0x7f1c9c09e560 msgr2=0x7f1c9c0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1ca29e6640 1 -- 172.21.15.37:0/4211925269 shutdown_connections 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.849+0000 7f1ca29e6640 1 -- 172.21.15.37:0/4211925269 wait complete. 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.850+0000 7f1ca29e6640 1 Processor -- start 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.851+0000 7f1ca29e6640 1 -- start start 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.851+0000 7f1ca29e6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c9c0d5b50 0x7f1c9c153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.851+0000 7f1ca29e6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c9c15f380 0x7f1c9c0c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.851+0000 7f1ca29e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c9c15f760 0x7f1c9c161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.851+0000 7f1ca29e6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c0d1280 con 0x7f1c9c15f380 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.851+0000 7f1ca29e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c0d1100 con 0x7f1c9c15f760 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.851+0000 7f1ca29e6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9c0d3840 con 0x7f1c9c0d5b50 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c9b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c9c15f380 0x7f1c9c0c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c9b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c9c15f380 0x7f1c9c0c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43792/0 (socket says 172.21.15.37:43792) 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c9b7fe640 1 -- 172.21.15.37:0/1883092192 learned_addr learned my addr 172.21.15.37:0/1883092192 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1ca0f83640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c9c15f760 0x7f1c9c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c9bfff640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c9c0d5b50 0x7f1c9c153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c9b7fe640 1 -- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c9c0d5b50 msgr2=0x7f1c9c153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c9b7fe640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c9c0d5b50 0x7f1c9c153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c9b7fe640 1 -- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c9c15f760 msgr2=0x7f1c9c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c9b7fe640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c9c15f760 0x7f1c9c161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c9b7fe640 1 -- 172.21.15.37:0/1883092192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c153fe0 con 0x7f1c9c15f380 2024-04-26T20:24:10.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1ca0f83640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c9c15f760 0x7f1c9c161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:10.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c9bfff640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c9c0d5b50 0x7f1c9c153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:10.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c9b7fe640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c9c15f380 0x7f1c9c0c89b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000c7f0 tx=0x7f1c9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:10.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c98ff9640 1 -- 172.21.15.37:0/1883092192 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90015070 con 0x7f1c9c15f380 2024-04-26T20:24:10.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c98ff9640 1 -- 172.21.15.37:0/1883092192 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c90007820 con 0x7f1c9c15f380 2024-04-26T20:24:10.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1c98ff9640 1 -- 172.21.15.37:0/1883092192 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90007bd0 con 0x7f1c9c15f380 2024-04-26T20:24:10.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1ca29e6640 1 -- 172.21.15.37:0/1883092192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9c0c9450 con 0x7f1c9c15f380 2024-04-26T20:24:10.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.852+0000 7f1ca29e6640 1 -- 172.21.15.37:0/1883092192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9c0c9a10 con 0x7f1c9c15f380 2024-04-26T20:24:10.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.853+0000 7f1ca29e6640 1 -- 172.21.15.37:0/1883092192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9c06a360 con 0x7f1c9c15f380 2024-04-26T20:24:10.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.854+0000 7f1c98ff9640 1 -- 172.21.15.37:0/1883092192 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f1c90018070 con 0x7f1c9c15f380 2024-04-26T20:24:10.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.854+0000 7f1c98ff9640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1c8806cbc0 0x7f1c8806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.854+0000 7f1c98ff9640 1 -- 172.21.15.37:0/1883092192 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1c9008c150 con 0x7f1c9c15f380 2024-04-26T20:24:10.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.855+0000 7f1c9bfff640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1c8806cbc0 0x7f1c8806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.855+0000 7f1c9bfff640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1c8806cbc0 0x7f1c8806f080 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00aa50 tx=0x7f1c8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:10.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:10.859+0000 7f1c98ff9640 1 -- 172.21.15.37:0/1883092192 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c9005a170 con 0x7f1c9c15f380 2024-04-26T20:24:11.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.046+0000 7f1ca29e6640 1 -- 172.21.15.37:0/1883092192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c9c15eaa0 con 0x7f1c9c15f380 2024-04-26T20:24:11.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.047+0000 7f1c98ff9640 1 -- 172.21.15.37:0/1883092192 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16638 (secure 0 0 0) 0x7f1c90059a70 con 0x7f1c9c15f380 2024-04-26T20:24:11.048 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:11.049 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":45,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:24:09.772733+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:24:08.047629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:24:08.708738+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:24:11.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 -- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1c8806cbc0 msgr2=0x7f1c8806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1c8806cbc0 0x7f1c8806f080 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00aa50 tx=0x7f1c8c008040 comp rx=0 tx=0).stop 2024-04-26T20:24:11.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 -- 172.21.15.37:0/1883092192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c9c15f380 msgr2=0x7f1c9c0c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c9c15f380 0x7f1c9c0c89b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000c7f0 tx=0x7f1c9000ccb0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 -- 172.21.15.37:0/1883092192 shutdown_connections 2024-04-26T20:24:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c9c15f760 0x7f1c9c161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c9c15f380 0x7f1c9c0c89b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1c8806cbc0 0x7f1c8806f080 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 --2- 172.21.15.37:0/1883092192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c9c0d5b50 0x7f1c9c153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 -- 172.21.15.37:0/1883092192 >> 172.21.15.37:0/1883092192 conn(0x7f1c9c09e560 msgr2=0x7f1c9c0b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 -- 172.21.15.37:0/1883092192 shutdown_connections 2024-04-26T20:24:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.049+0000 7f1ca29e6640 1 -- 172.21.15.37:0/1883092192 wait complete. 2024-04-26T20:24:11.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:24:11.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.164+0000 7f3397f59640 1 Processor -- start 2024-04-26T20:24:11.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.164+0000 7f3397f59640 1 -- start start 2024-04-26T20:24:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.165+0000 7f3397f59640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f33901610c0 0x7f33900d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.165+0000 7f3397f59640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f33900d2150 0x7f33900d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.165+0000 7f3397f59640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3390161510 con 0x7f3390160800 2024-04-26T20:24:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.165+0000 7f3397f59640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f33900d25c0 con 0x7f33901610c0 2024-04-26T20:24:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.165+0000 7f3397f59640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f33900d0930 con 0x7f33900d2150 2024-04-26T20:24:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.165+0000 7f33954f4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f33901610c0 0x7f33900d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.165+0000 7f3395cf5640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3390160800 0x7f33900d5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59714/0 (socket says 172.21.15.37:59714) 2024-04-26T20:24:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.165+0000 7f3395cf5640 1 -- 172.21.15.37:0/1574424946 learned_addr learned my addr 172.21.15.37:0/1574424946 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.165+0000 7f33964f6640 1 --2- 172.21.15.37:0/1574424946 >> v2:172.21.15.134:3300/0 conn(0x7f33900d2150 0x7f33900d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.166+0000 7f33954f4640 1 -- 172.21.15.37:0/1574424946 >> v2:172.21.15.134:3300/0 conn(0x7f33900d2150 msgr2=0x7f33900d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:11.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.166+0000 7f33954f4640 1 --2- 172.21.15.37:0/1574424946 >> v2:172.21.15.134:3300/0 conn(0x7f33900d2150 0x7f33900d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.166+0000 7f33954f4640 1 -- 172.21.15.37:0/1574424946 >> v1:172.21.15.37:6789/0 conn(0x7f3390160800 legacy=0x7f33900d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.166+0000 7f33954f4640 1 -- 172.21.15.37:0/1574424946 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33900d0c00 con 0x7f33901610c0 2024-04-26T20:24:11.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.166+0000 7f3394cf3640 1 -- 172.21.15.37:0/1574424946 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 330037217 0 0) 0x7f3390161510 con 0x7f3390160800 2024-04-26T20:24:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.166+0000 7f33954f4640 1 --2- 172.21.15.37:0/1574424946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33901610c0 0x7f33900d1c10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f33900d4d40 tx=0x7f338402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ef14c722bf26e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.167+0000 7f3394cf3640 1 -- 172.21.15.37:0/1574424946 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384037070 con 0x7f33901610c0 2024-04-26T20:24:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.167+0000 7f3394cf3640 1 -- 172.21.15.37:0/1574424946 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3384032440 con 0x7f33901610c0 2024-04-26T20:24:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.167+0000 7f3394cf3640 1 -- 172.21.15.37:0/1574424946 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384032810 con 0x7f33901610c0 2024-04-26T20:24:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.167+0000 7f3397f59640 1 -- 172.21.15.37:0/1574424946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33901610c0 msgr2=0x7f33900d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.167+0000 7f3397f59640 1 --2- 172.21.15.37:0/1574424946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33901610c0 0x7f33900d1c10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f33900d4d40 tx=0x7f338402a1a0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.167+0000 7f3397f59640 1 -- 172.21.15.37:0/1574424946 shutdown_connections 2024-04-26T20:24:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.167+0000 7f3397f59640 1 --2- 172.21.15.37:0/1574424946 >> v2:172.21.15.134:3300/0 conn(0x7f33900d2150 0x7f33900d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.167+0000 7f3397f59640 1 --2- 172.21.15.37:0/1574424946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33901610c0 0x7f33900d1c10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.167+0000 7f3397f59640 1 -- 172.21.15.37:0/1574424946 >> 172.21.15.37:0/1574424946 conn(0x7f339009e550 msgr2=0x7f33900d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.167+0000 7f3397f59640 1 -- 172.21.15.37:0/1574424946 shutdown_connections 2024-04-26T20:24:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.167+0000 7f3397f59640 1 -- 172.21.15.37:0/1574424946 wait complete. 2024-04-26T20:24:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.168+0000 7f3397f59640 1 Processor -- start 2024-04-26T20:24:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.168+0000 7f3397f59640 1 -- start start 2024-04-26T20:24:11.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3397f59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33900d2150 0x7f339006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3397f59640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3390160800 0x7f33900b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3397f59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33901610c0 0x7f33900b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3397f59640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f33900d2e10 con 0x7f33900d2150 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3397f59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f33900d11c0 con 0x7f33901610c0 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3397f59640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f33900d30f0 con 0x7f3390160800 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3395cf5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33900d2150 0x7f339006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3395cf5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33900d2150 0x7f339006d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43808/0 (socket says 172.21.15.37:43808) 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3395cf5640 1 -- 172.21.15.37:0/130128614 learned_addr learned my addr 172.21.15.37:0/130128614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f33954f4640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3390160800 0x7f33900b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f33964f6640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33901610c0 0x7f33900b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3395cf5640 1 -- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3390160800 msgr2=0x7f33900b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3395cf5640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3390160800 0x7f33900b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3395cf5640 1 -- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33901610c0 msgr2=0x7f33900b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.169+0000 7f3395cf5640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33901610c0 0x7f33900b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.170+0000 7f3395cf5640 1 -- 172.21.15.37:0/130128614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33900b8b00 con 0x7f33900d2150 2024-04-26T20:24:11.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.170+0000 7f33964f6640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33901610c0 0x7f33900b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:11.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.170+0000 7f33954f4640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3390160800 0x7f33900b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:11.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.170+0000 7f3395cf5640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33900d2150 0x7f339006d500 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f338800c4d0 tx=0x7f338800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:11.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.170+0000 7f338e7fc640 1 -- 172.21.15.37:0/130128614 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388018070 con 0x7f33900d2150 2024-04-26T20:24:11.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.170+0000 7f338e7fc640 1 -- 172.21.15.37:0/130128614 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3388007500 con 0x7f33900d2150 2024-04-26T20:24:11.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.170+0000 7f3397f59640 1 -- 172.21.15.37:0/130128614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33900b2c00 con 0x7f33900d2150 2024-04-26T20:24:11.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.170+0000 7f338e7fc640 1 -- 172.21.15.37:0/130128614 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388004a20 con 0x7f33900d2150 2024-04-26T20:24:11.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.170+0000 7f3397f59640 1 -- 172.21.15.37:0/130128614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33900b9080 con 0x7f33900d2150 2024-04-26T20:24:11.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.171+0000 7f3397f59640 1 -- 172.21.15.37:0/130128614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339006a360 con 0x7f33900d2150 2024-04-26T20:24:11.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.171+0000 7f338e7fc640 1 -- 172.21.15.37:0/130128614 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f338801b070 con 0x7f33900d2150 2024-04-26T20:24:11.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.172+0000 7f338e7fc640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f337c06cbc0 0x7f337c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.172+0000 7f338e7fc640 1 -- 172.21.15.37:0/130128614 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f338808b3a0 con 0x7f33900d2150 2024-04-26T20:24:11.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.172+0000 7f33954f4640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f337c06cbc0 0x7f337c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.173+0000 7f33954f4640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f337c06cbc0 0x7f337c06f080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3384034000 tx=0x7f338402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:11.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.176+0000 7f338e7fc640 1 -- 172.21.15.37:0/130128614 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f33880593a0 con 0x7f33900d2150 2024-04-26T20:24:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.401+0000 7f3397f59640 1 -- 172.21.15.37:0/130128614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f33900d2960 con 0x7f33900d2150 2024-04-26T20:24:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.408+0000 7f338e7fc640 1 -- 172.21.15.37:0/130128614 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7f3388058ca0 con 0x7f33900d2150 2024-04-26T20:24:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.411+0000 7f3397f59640 1 -- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f337c06cbc0 msgr2=0x7f337c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.411+0000 7f3397f59640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f337c06cbc0 0x7f337c06f080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3384034000 tx=0x7f338402e000 comp rx=0 tx=0).stop 2024-04-26T20:24:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.411+0000 7f3397f59640 1 -- 172.21.15.37:0/130128614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33900d2150 msgr2=0x7f339006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.411+0000 7f3397f59640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33900d2150 0x7f339006d500 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f338800c4d0 tx=0x7f338800c990 comp rx=0 tx=0).stop 2024-04-26T20:24:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.411+0000 7f3397f59640 1 -- 172.21.15.37:0/130128614 shutdown_connections 2024-04-26T20:24:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.411+0000 7f3397f59640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33901610c0 0x7f33900b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.411+0000 7f3397f59640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f337c06cbc0 0x7f337c06f080 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.411+0000 7f3397f59640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3390160800 0x7f33900b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.411+0000 7f3397f59640 1 --2- 172.21.15.37:0/130128614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33900d2150 0x7f339006d500 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.411+0000 7f3397f59640 1 -- 172.21.15.37:0/130128614 >> 172.21.15.37:0/130128614 conn(0x7f339009e550 msgr2=0x7f339015e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.412+0000 7f3397f59640 1 -- 172.21.15.37:0/130128614 shutdown_connections 2024-04-26T20:24:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.412+0000 7f3397f59640 1 -- 172.21.15.37:0/130128614 wait complete. 2024-04-26T20:24:11.424 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:24:11.424 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:24:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.527+0000 7f6a24538640 1 Processor -- start 2024-04-26T20:24:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.527+0000 7f6a24538640 1 -- start start 2024-04-26T20:24:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a24538640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6a1c15d460 0x7f6a1c06e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a24538640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6a1c073c40 0x7f6a1c0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a24538640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a1c06ece0 con 0x7f6a1c0cd100 2024-04-26T20:24:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a24538640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6a1c0add80 con 0x7f6a1c15d460 2024-04-26T20:24:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a24538640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6a1c06efc0 con 0x7f6a1c073c40 2024-04-26T20:24:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a222d4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6a1c15d460 0x7f6a1c06e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a22ad5640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6a1c0cd100 0x7f6a1c0cb3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59716/0 (socket says 172.21.15.37:59716) 2024-04-26T20:24:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a22ad5640 1 -- 172.21.15.37:0/640706575 learned_addr learned my addr 172.21.15.37:0/640706575 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a21ad3640 1 --2- 172.21.15.37:0/640706575 >> v2:172.21.15.134:3300/0 conn(0x7f6a1c073c40 0x7f6a1c0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a222d4640 1 -- 172.21.15.37:0/640706575 >> v2:172.21.15.134:3300/0 conn(0x7f6a1c073c40 msgr2=0x7f6a1c0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a222d4640 1 --2- 172.21.15.37:0/640706575 >> v2:172.21.15.134:3300/0 conn(0x7f6a1c073c40 0x7f6a1c0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a222d4640 1 -- 172.21.15.37:0/640706575 >> v1:172.21.15.37:6789/0 conn(0x7f6a1c0cd100 legacy=0x7f6a1c0cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a222d4640 1 -- 172.21.15.37:0/640706575 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1c06f2a0 con 0x7f6a1c15d460 2024-04-26T20:24:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a212d2640 1 -- 172.21.15.37:0/640706575 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3489295995 0 0) 0x7f6a1c06ece0 con 0x7f6a1c0cd100 2024-04-26T20:24:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.528+0000 7f6a21ad3640 1 --2- 172.21.15.37:0/640706575 >> v2:172.21.15.134:3300/0 conn(0x7f6a1c073c40 0x7f6a1c0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.529+0000 7f6a222d4640 1 --2- 172.21.15.37:0/640706575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1c15d460 0x7f6a1c06e700 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c073210 tx=0x7f6a1802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd66cb2f99cbda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:11.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.529+0000 7f6a212d2640 1 -- 172.21.15.37:0/640706575 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18037070 con 0x7f6a1c15d460 2024-04-26T20:24:11.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.529+0000 7f6a212d2640 1 -- 172.21.15.37:0/640706575 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a18032440 con 0x7f6a1c15d460 2024-04-26T20:24:11.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.529+0000 7f6a212d2640 1 -- 172.21.15.37:0/640706575 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18032810 con 0x7f6a1c15d460 2024-04-26T20:24:11.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.530+0000 7f6a24538640 1 -- 172.21.15.37:0/640706575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1c15d460 msgr2=0x7f6a1c06e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.530+0000 7f6a24538640 1 --2- 172.21.15.37:0/640706575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1c15d460 0x7f6a1c06e700 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c073210 tx=0x7f6a1802a230 comp rx=0 tx=0).stop 2024-04-26T20:24:11.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.530+0000 7f6a24538640 1 -- 172.21.15.37:0/640706575 shutdown_connections 2024-04-26T20:24:11.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.530+0000 7f6a24538640 1 --2- 172.21.15.37:0/640706575 >> v2:172.21.15.134:3300/0 conn(0x7f6a1c073c40 0x7f6a1c0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.530+0000 7f6a24538640 1 --2- 172.21.15.37:0/640706575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1c15d460 0x7f6a1c06e700 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.530+0000 7f6a24538640 1 -- 172.21.15.37:0/640706575 >> 172.21.15.37:0/640706575 conn(0x7f6a1c0d0d70 msgr2=0x7f6a1c0d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.530+0000 7f6a24538640 1 -- 172.21.15.37:0/640706575 shutdown_connections 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.530+0000 7f6a24538640 1 -- 172.21.15.37:0/640706575 wait complete. 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.531+0000 7f6a24538640 1 Processor -- start 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.531+0000 7f6a24538640 1 -- start start 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a24538640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1c073c40 0x7f6a1c153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a24538640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a1c0cd100 0x7f6a1c09e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a24538640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a1c15d460 0x7f6a1c09e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a24538640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c15d9c0 con 0x7f6a1c073c40 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a24538640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c15d840 con 0x7f6a1c15d460 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a24538640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6a1c0740a0 con 0x7f6a1c0cd100 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a222d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1c073c40 0x7f6a1c153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a222d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1c073c40 0x7f6a1c153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43834/0 (socket says 172.21.15.37:43834) 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a222d4640 1 -- 172.21.15.37:0/504451046 learned_addr learned my addr 172.21.15.37:0/504451046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a22ad5640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a1c15d460 0x7f6a1c09e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a21ad3640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a1c0cd100 0x7f6a1c09e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a222d4640 1 -- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a1c0cd100 msgr2=0x7f6a1c09e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a222d4640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a1c0cd100 0x7f6a1c09e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a222d4640 1 -- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a1c15d460 msgr2=0x7f6a1c09e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a222d4640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a1c15d460 0x7f6a1c09e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.532+0000 7f6a222d4640 1 -- 172.21.15.37:0/504451046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1c09c740 con 0x7f6a1c073c40 2024-04-26T20:24:11.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.533+0000 7f6a222d4640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1c073c40 0x7f6a1c153fa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009890 tx=0x7f6a180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:11.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.533+0000 7f6a12ffd640 1 -- 172.21.15.37:0/504451046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18037040 con 0x7f6a1c073c40 2024-04-26T20:24:11.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.533+0000 7f6a12ffd640 1 -- 172.21.15.37:0/504451046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a18042070 con 0x7f6a1c073c40 2024-04-26T20:24:11.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.533+0000 7f6a24538640 1 -- 172.21.15.37:0/504451046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1c09ca20 con 0x7f6a1c073c40 2024-04-26T20:24:11.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.533+0000 7f6a12ffd640 1 -- 172.21.15.37:0/504451046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18030a50 con 0x7f6a1c073c40 2024-04-26T20:24:11.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.533+0000 7f6a24538640 1 -- 172.21.15.37:0/504451046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a1c09cfe0 con 0x7f6a1c073c40 2024-04-26T20:24:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.534+0000 7f6a24538640 1 -- 172.21.15.37:0/504451046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a1c06a360 con 0x7f6a1c073c40 2024-04-26T20:24:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.535+0000 7f6a12ffd640 1 -- 172.21.15.37:0/504451046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f6a18030bf0 con 0x7f6a1c073c40 2024-04-26T20:24:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.535+0000 7f6a12ffd640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6a0006cbc0 0x7f6a0006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.535+0000 7f6a12ffd640 1 -- 172.21.15.37:0/504451046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f6a180ac9f0 con 0x7f6a1c073c40 2024-04-26T20:24:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.535+0000 7f6a21ad3640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6a0006cbc0 0x7f6a0006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.536+0000 7f6a21ad3640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6a0006cbc0 0x7f6a0006f080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c15b520 tx=0x7f6a0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.539+0000 7f6a12ffd640 1 -- 172.21.15.37:0/504451046 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a18045360 con 0x7f6a1c073c40 2024-04-26T20:24:11.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:11.764+0000 7f6a24538640 1 -- 172.21.15.37:0/504451046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6a1c06fe10 con 0x7f6a1c073c40 2024-04-26T20:24:12.381 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-26T20:24:12.381+0000 7f865e80b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:24:12.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.415+0000 7f6a12ffd640 1 -- 172.21.15.37:0/504451046 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) ==== 122+0+0 (secure 0 0 0) 0x7f6a1807aad0 con 0x7f6a1c073c40 2024-04-26T20:24:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 -- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6a0006cbc0 msgr2=0x7f6a0006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6a0006cbc0 0x7f6a0006f080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c15b520 tx=0x7f6a0c008040 comp rx=0 tx=0).stop 2024-04-26T20:24:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 -- 172.21.15.37:0/504451046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1c073c40 msgr2=0x7f6a1c153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1c073c40 0x7f6a1c153fa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009890 tx=0x7f6a180046c0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 -- 172.21.15.37:0/504451046 shutdown_connections 2024-04-26T20:24:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a1c15d460 0x7f6a1c09e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f6a0006cbc0 0x7f6a0006f080 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a1c0cd100 0x7f6a1c09e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 --2- 172.21.15.37:0/504451046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a1c073c40 0x7f6a1c153fa0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 -- 172.21.15.37:0/504451046 >> 172.21.15.37:0/504451046 conn(0x7f6a1c0d0d70 msgr2=0x7f6a1c071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:12.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 -- 172.21.15.37:0/504451046 shutdown_connections 2024-04-26T20:24:12.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.418+0000 7f6a24538640 1 -- 172.21.15.37:0/504451046 wait complete. 2024-04-26T20:24:12.432 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:24:12.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:24:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.538+0000 7fb1c4808640 1 Processor -- start 2024-04-26T20:24:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.538+0000 7fb1c4808640 1 -- start start 2024-04-26T20:24:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1c4808640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb1c00b1f90 0x7fb1c00d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1c4808640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c00b2c80 0x7fb1c00b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1c4808640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1c00cf740 con 0x7fb1c0160a00 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1c4808640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb1c00b2400 con 0x7fb1c00b1f90 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1c4808640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c00cf080 con 0x7fb1c00b2c80 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1bdd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb1c00b1f90 0x7fb1c00d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1be59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb1c0160a00 0x7fb1c00cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59724/0 (socket says 172.21.15.37:59724) 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1be59c640 1 -- 172.21.15.37:0/3012867341 learned_addr learned my addr 172.21.15.37:0/3012867341 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1bed9d640 1 --2- 172.21.15.37:0/3012867341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c00b2c80 0x7fb1c00b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1bdd9b640 1 -- 172.21.15.37:0/3012867341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c00b2c80 msgr2=0x7fb1c00b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1bdd9b640 1 --2- 172.21.15.37:0/3012867341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c00b2c80 0x7fb1c00b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1bdd9b640 1 -- 172.21.15.37:0/3012867341 >> v1:172.21.15.37:6789/0 conn(0x7fb1c0160a00 legacy=0x7fb1c00cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1bdd9b640 1 -- 172.21.15.37:0/3012867341 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c00ce7e0 con 0x7fb1c00b1f90 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1bd59a640 1 -- 172.21.15.37:0/3012867341 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3852841419 0 0) 0x7fb1c00cf740 con 0x7fb1c0160a00 2024-04-26T20:24:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.539+0000 7fb1bed9d640 1 --2- 172.21.15.37:0/3012867341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c00b2c80 0x7fb1c00b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.540+0000 7fb1bdd9b640 1 --2- 172.21.15.37:0/3012867341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c00b1f90 0x7fb1c00d00d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40097a0 tx=0x7fb1b402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ddf4628437c99a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.540+0000 7fb1bd59a640 1 -- 172.21.15.37:0/3012867341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4037070 con 0x7fb1c00b1f90 2024-04-26T20:24:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.540+0000 7fb1bd59a640 1 -- 172.21.15.37:0/3012867341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b4032440 con 0x7fb1c00b1f90 2024-04-26T20:24:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.540+0000 7fb1bd59a640 1 -- 172.21.15.37:0/3012867341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4032810 con 0x7fb1c00b1f90 2024-04-26T20:24:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.541+0000 7fb1c4808640 1 -- 172.21.15.37:0/3012867341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c00b1f90 msgr2=0x7fb1c00d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.541+0000 7fb1c4808640 1 --2- 172.21.15.37:0/3012867341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c00b1f90 0x7fb1c00d00d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40097a0 tx=0x7fb1b402a210 comp rx=0 tx=0).stop 2024-04-26T20:24:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.541+0000 7fb1c4808640 1 -- 172.21.15.37:0/3012867341 shutdown_connections 2024-04-26T20:24:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.541+0000 7fb1c4808640 1 --2- 172.21.15.37:0/3012867341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c00b2c80 0x7fb1c00b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.541+0000 7fb1c4808640 1 --2- 172.21.15.37:0/3012867341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c00b1f90 0x7fb1c00d00d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.541+0000 7fb1c4808640 1 -- 172.21.15.37:0/3012867341 >> 172.21.15.37:0/3012867341 conn(0x7fb1c009ba50 msgr2=0x7fb1c009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:12.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.541+0000 7fb1c4808640 1 -- 172.21.15.37:0/3012867341 shutdown_connections 2024-04-26T20:24:12.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.541+0000 7fb1c4808640 1 -- 172.21.15.37:0/3012867341 wait complete. 2024-04-26T20:24:12.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.542+0000 7fb1c4808640 1 Processor -- start 2024-04-26T20:24:12.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.542+0000 7fb1c4808640 1 -- start start 2024-04-26T20:24:12.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1c4808640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c00b1f90 0x7fb1c014b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1c4808640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c00b2c80 0x7fb1c00712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1c4808640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c0160a00 0x7fb1c0071820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1c4808640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0160de0 con 0x7fb1c00b2c80 2024-04-26T20:24:12.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1c4808640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c00ceda0 con 0x7fb1c00b1f90 2024-04-26T20:24:12.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1c4808640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c00b2690 con 0x7fb1c0160a00 2024-04-26T20:24:12.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c00b2c80 0x7fb1c00712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c00b2c80 0x7fb1c00712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43850/0 (socket says 172.21.15.37:43850) 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1bdd9b640 1 -- 172.21.15.37:0/482211744 learned_addr learned my addr 172.21.15.37:0/482211744 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1bdd9b640 1 -- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c0160a00 msgr2=0x7fb1c0071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1be59c640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c00b1f90 0x7fb1c014b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1bed9d640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c0160a00 0x7fb1c0071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1bdd9b640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c0160a00 0x7fb1c0071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1bdd9b640 1 -- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c00b1f90 msgr2=0x7fb1c014b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1bdd9b640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c00b1f90 0x7fb1c014b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1bdd9b640 1 -- 172.21.15.37:0/482211744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c014cf50 con 0x7fb1c00b2c80 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1bed9d640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c0160a00 0x7fb1c0071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.543+0000 7fb1be59c640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c00b1f90 0x7fb1c014b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.544+0000 7fb1bdd9b640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c00b2c80 0x7fb1c00712e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40097a0 tx=0x7fb1b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.544+0000 7fb1aeffd640 1 -- 172.21.15.37:0/482211744 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4037050 con 0x7fb1c00b2c80 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.544+0000 7fb1aeffd640 1 -- 172.21.15.37:0/482211744 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b4038040 con 0x7fb1c00b2c80 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.544+0000 7fb1c4808640 1 -- 172.21.15.37:0/482211744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c014d230 con 0x7fb1c00b2c80 2024-04-26T20:24:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.544+0000 7fb1aeffd640 1 -- 172.21.15.37:0/482211744 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4004490 con 0x7fb1c00b2c80 2024-04-26T20:24:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.544+0000 7fb1c4808640 1 -- 172.21.15.37:0/482211744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c014d7f0 con 0x7fb1c00b2c80 2024-04-26T20:24:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.545+0000 7fb1c4808640 1 -- 172.21.15.37:0/482211744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c006a360 con 0x7fb1c00b2c80 2024-04-26T20:24:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.545+0000 7fb1aeffd640 1 -- 172.21.15.37:0/482211744 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fb1b4030980 con 0x7fb1c00b2c80 2024-04-26T20:24:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.546+0000 7fb1aeffd640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb1a006cbc0 0x7fb1a006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.546+0000 7fb1aeffd640 1 -- 172.21.15.37:0/482211744 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fb1b40acba0 con 0x7fb1c00b2c80 2024-04-26T20:24:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.546+0000 7fb1be59c640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb1a006cbc0 0x7fb1a006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.547+0000 7fb1be59c640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb1a006cbc0 0x7fb1a006f080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0000f60 tx=0x7fb1b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:12.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.550+0000 7fb1aeffd640 1 -- 172.21.15.37:0/482211744 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb1b407adf0 con 0x7fb1c00b2c80 2024-04-26T20:24:12.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.775+0000 7fb1c4808640 1 -- 172.21.15.37:0/482211744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1c00b3540 con 0x7fb1c00b2c80 2024-04-26T20:24:12.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.775+0000 7fb1aeffd640 1 -- 172.21.15.37:0/482211744 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5178 (secure 0 0 0) 0x7fb1b407a6f0 con 0x7fb1c00b2c80 2024-04-26T20:24:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:12.777 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/689917778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":689917778},{"type":"v1","addr":"172.21.15.134:6836","nonce":689917778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5362,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/497972484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":497972484},{"type":"v1","addr":"172.21.15.134:6837","nonce":497972484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/221502372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":221502372},{"type":"v1","addr":"172.21.15.37:6833","nonce":221502372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:24:09.772695+0000","modified":"2024-04-26T20:24:11.765632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1915220099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1915220099},{"type":"v1","addr":"172.21.15.37:6835","nonce":1915220099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5688,"qdb_cluster":[5688]},"id":3}]} 2024-04-26T20:24:12.777 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 -- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb1a006cbc0 msgr2=0x7fb1a006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb1a006cbc0 0x7fb1a006f080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0000f60 tx=0x7fb1b0009040 comp rx=0 tx=0).stop 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 -- 172.21.15.37:0/482211744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c00b2c80 msgr2=0x7fb1c00712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c00b2c80 0x7fb1c00712e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40097a0 tx=0x7fb1b40046c0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 -- 172.21.15.37:0/482211744 shutdown_connections 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fb1a006cbc0 0x7fb1a006f080 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1c0160a00 0x7fb1c0071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c00b2c80 0x7fb1c00712e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 --2- 172.21.15.37:0/482211744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c00b1f90 0x7fb1c014b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 -- 172.21.15.37:0/482211744 >> 172.21.15.37:0/482211744 conn(0x7fb1c009ba50 msgr2=0x7fb1c009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 -- 172.21.15.37:0/482211744 shutdown_connections 2024-04-26T20:24:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.777+0000 7fb1c4808640 1 -- 172.21.15.37:0/482211744 wait complete. 2024-04-26T20:24:12.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:24:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.889+0000 7f4edc052640 1 Processor -- start 2024-04-26T20:24:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.890+0000 7f4edc052640 1 -- start start 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.890+0000 7f4edc052640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed4162b40 0x7f4ed40d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.890+0000 7f4edc052640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4ed40d3c80 0x7f4ed40d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.890+0000 7f4edc052640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed40d0210 con 0x7f4ed40d3820 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.890+0000 7f4edc052640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed40d04f0 con 0x7f4ed40d3c80 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.890+0000 7f4edc052640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed40d07d0 con 0x7f4ed4162b40 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.891+0000 7f4eda5ef640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4ed40d3c80 0x7f4ed40d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.891+0000 7f4ed95ed640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4ed40d3820 0x7f4ed40d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59732/0 (socket says 172.21.15.37:59732) 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.891+0000 7f4eda5ef640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4ed40d3c80 0x7f4ed40d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43852/0 (socket says 172.21.15.37:43852) 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.891+0000 7f4ed95ed640 1 -- 172.21.15.37:0/3794526290 learned_addr learned my addr 172.21.15.37:0/3794526290 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.891+0000 7f4ed9dee640 1 --2- 172.21.15.37:0/3794526290 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed4162b40 0x7f4ed40d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.891+0000 7f4eda5ef640 1 -- 172.21.15.37:0/3794526290 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed4162b40 msgr2=0x7f4ed40d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.891+0000 7f4eda5ef640 1 --2- 172.21.15.37:0/3794526290 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed4162b40 0x7f4ed40d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.891+0000 7f4eda5ef640 1 -- 172.21.15.37:0/3794526290 >> v1:172.21.15.37:6789/0 conn(0x7f4ed40d3820 legacy=0x7f4ed40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.891+0000 7f4eda5ef640 1 -- 172.21.15.37:0/3794526290 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed40d0ab0 con 0x7f4ed40d3c80 2024-04-26T20:24:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.891+0000 7f4ed8dec640 1 -- 172.21.15.37:0/3794526290 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2052680810 0 0) 0x7f4ed40d0210 con 0x7f4ed40d3820 2024-04-26T20:24:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.891+0000 7f4ed9dee640 1 --2- 172.21.15.37:0/3794526290 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed4162b40 0x7f4ed40d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.892+0000 7f4eda5ef640 1 --2- 172.21.15.37:0/3794526290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40d3c80 0x7f4ed40d1f60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009d90 tx=0x7f4ecc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fecb0df58eee819 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.892+0000 7f4ed8dec640 1 -- 172.21.15.37:0/3794526290 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc007740 con 0x7f4ed40d3c80 2024-04-26T20:24:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.892+0000 7f4ed8dec640 1 -- 172.21.15.37:0/3794526290 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ecc0078e0 con 0x7f4ed40d3c80 2024-04-26T20:24:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.892+0000 7f4ed8dec640 1 -- 172.21.15.37:0/3794526290 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc007cb0 con 0x7f4ed40d3c80 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.892+0000 7f4edc052640 1 -- 172.21.15.37:0/3794526290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40d3c80 msgr2=0x7f4ed40d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.892+0000 7f4edc052640 1 --2- 172.21.15.37:0/3794526290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40d3c80 0x7f4ed40d1f60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009d90 tx=0x7f4ecc02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.892+0000 7f4edc052640 1 -- 172.21.15.37:0/3794526290 shutdown_connections 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.892+0000 7f4edc052640 1 --2- 172.21.15.37:0/3794526290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40d3c80 0x7f4ed40d1f60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.892+0000 7f4edc052640 1 --2- 172.21.15.37:0/3794526290 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed4162b40 0x7f4ed40d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.892+0000 7f4edc052640 1 -- 172.21.15.37:0/3794526290 >> 172.21.15.37:0/3794526290 conn(0x7f4ed409e550 msgr2=0x7f4ed40d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.893+0000 7f4edc052640 1 -- 172.21.15.37:0/3794526290 shutdown_connections 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.893+0000 7f4edc052640 1 -- 172.21.15.37:0/3794526290 wait complete. 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.894+0000 7f4edc052640 1 Processor -- start 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.894+0000 7f4edc052640 1 -- start start 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.894+0000 7f4edc052640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40d3820 0x7f4ed40c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.894+0000 7f4edc052640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed40d3c80 0x7f4ed40b8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.894+0000 7f4edc052640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ed4162b40 0x7f4ed4158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.894+0000 7f4edc052640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed40b5bb0 con 0x7f4ed40d3820 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.894+0000 7f4edc052640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed40d4110 con 0x7f4ed4162b40 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.894+0000 7f4edc052640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed40d6110 con 0x7f4ed40d3c80 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4ed9dee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40d3820 0x7f4ed40c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4ed9dee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40d3820 0x7f4ed40c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43856/0 (socket says 172.21.15.37:43856) 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4ed9dee640 1 -- 172.21.15.37:0/1430563675 learned_addr learned my addr 172.21.15.37:0/1430563675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4ed95ed640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed40d3c80 0x7f4ed40b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4ed9dee640 1 -- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed40d3c80 msgr2=0x7f4ed40b8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4eda5ef640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ed4162b40 0x7f4ed4158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4ed9dee640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed40d3c80 0x7f4ed40b8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4ed9dee640 1 -- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ed4162b40 msgr2=0x7f4ed4158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4ed9dee640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ed4162b40 0x7f4ed4158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4ed9dee640 1 -- 172.21.15.37:0/1430563675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed40cee10 con 0x7f4ed40d3820 2024-04-26T20:24:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4ed95ed640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed40d3c80 0x7f4ed40b8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4eda5ef640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ed4162b40 0x7f4ed4158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4ed9dee640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40d3820 0x7f4ed40c88d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000e7e0 tx=0x7f4ed000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4eca7fc640 1 -- 172.21.15.37:0/1430563675 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed000f040 con 0x7f4ed40d3820 2024-04-26T20:24:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4eca7fc640 1 -- 172.21.15.37:0/1430563675 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed0004030 con 0x7f4ed40d3820 2024-04-26T20:24:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4eca7fc640 1 -- 172.21.15.37:0/1430563675 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0004400 con 0x7f4ed40d3820 2024-04-26T20:24:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4edc052640 1 -- 172.21.15.37:0/1430563675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed40ce280 con 0x7f4ed40d3820 2024-04-26T20:24:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.895+0000 7f4edc052640 1 -- 172.21.15.37:0/1430563675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed40b2210 con 0x7f4ed40d3820 2024-04-26T20:24:12.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.896+0000 7f4edc052640 1 -- 172.21.15.37:0/1430563675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed406a360 con 0x7f4ed40d3820 2024-04-26T20:24:12.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.897+0000 7f4eca7fc640 1 -- 172.21.15.37:0/1430563675 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f4ed000c6d0 con 0x7f4ed40d3820 2024-04-26T20:24:12.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.897+0000 7f4eca7fc640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4eb806cbc0 0x7f4eb806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.897+0000 7f4eca7fc640 1 -- 172.21.15.37:0/1430563675 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f4ed008bef0 con 0x7f4ed40d3820 2024-04-26T20:24:12.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.898+0000 7f4ed95ed640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4eb806cbc0 0x7f4eb806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.898+0000 7f4ed95ed640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4eb806cbc0 0x7f4eb806f080 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40b9ec0 tx=0x7f4ec4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:12.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:12.902+0000 7f4eca7fc640 1 -- 172.21.15.37:0/1430563675 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ed0017020 con 0x7f4ed40d3820 2024-04-26T20:24:13.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.087+0000 7f4edc052640 1 -- 172.21.15.37:0/1430563675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ed406eb60 con 0x7f4ed40d3820 2024-04-26T20:24:13.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.088+0000 7f4eca7fc640 1 -- 172.21.15.37:0/1430563675 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16625 (secure 0 0 0) 0x7f4ed0059e70 con 0x7f4ed40d3820 2024-04-26T20:24:13.089 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:13.089 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":46,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:24:11.405646+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:24:08.047629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:24:08.708738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:24:13.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 -- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4eb806cbc0 msgr2=0x7f4eb806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4eb806cbc0 0x7f4eb806f080 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40b9ec0 tx=0x7f4ec4009040 comp rx=0 tx=0).stop 2024-04-26T20:24:13.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 -- 172.21.15.37:0/1430563675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40d3820 msgr2=0x7f4ed40c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40d3820 0x7f4ed40c88d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000e7e0 tx=0x7f4ed000eca0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 -- 172.21.15.37:0/1430563675 shutdown_connections 2024-04-26T20:24:13.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ed4162b40 0x7f4ed4158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f4eb806cbc0 0x7f4eb806f080 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ed40d3c80 0x7f4ed40b8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 --2- 172.21.15.37:0/1430563675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40d3820 0x7f4ed40c88d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 -- 172.21.15.37:0/1430563675 >> 172.21.15.37:0/1430563675 conn(0x7f4ed409e550 msgr2=0x7f4ed406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:13.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 -- 172.21.15.37:0/1430563675 shutdown_connections 2024-04-26T20:24:13.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.090+0000 7f4edc052640 1 -- 172.21.15.37:0/1430563675 wait complete. 2024-04-26T20:24:13.102 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:24:13.103 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:24:13.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.202+0000 7f97dc27b640 1 Processor -- start 2024-04-26T20:24:13.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.202+0000 7f97dc27b640 1 -- start start 2024-04-26T20:24:13.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97dc27b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97d415fc10 0x7f97d41600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97dc27b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f97d40ce9b0 0x7f97d40cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97dc27b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d4072e30 con 0x7f97d4159a90 2024-04-26T20:24:13.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97dc27b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f97d4073110 con 0x7f97d415fc10 2024-04-26T20:24:13.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97dc27b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f97d40733f0 con 0x7f97d40ce9b0 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97d9816640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97d415fc10 0x7f97d41600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97d9816640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97d415fc10 0x7f97d41600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43870/0 (socket says 172.21.15.37:43870) 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97d9816640 1 -- 172.21.15.37:0/2114704734 learned_addr learned my addr 172.21.15.37:0/2114704734 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97da818640 1 --2- 172.21.15.37:0/2114704734 >> v2:172.21.15.134:3300/0 conn(0x7f97d40ce9b0 0x7f97d40cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97d9816640 1 -- 172.21.15.37:0/2114704734 >> v2:172.21.15.134:3300/0 conn(0x7f97d40ce9b0 msgr2=0x7f97d40cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97d9816640 1 --2- 172.21.15.37:0/2114704734 >> v2:172.21.15.134:3300/0 conn(0x7f97d40ce9b0 0x7f97d40cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97d9816640 1 -- 172.21.15.37:0/2114704734 >> v1:172.21.15.134:6789/0 conn(0x7f97d4159a90 legacy=0x7f97d406fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.203+0000 7f97d9816640 1 -- 172.21.15.37:0/2114704734 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d40736d0 con 0x7f97d415fc10 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.204+0000 7f97d9816640 1 --2- 172.21.15.37:0/2114704734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d415fc10 0x7f97d41600c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f97d4074710 tx=0x7f97c402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deb8069a8c9bcfc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.204+0000 7f97d9015640 1 -- 172.21.15.37:0/2114704734 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4037070 con 0x7f97d415fc10 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.204+0000 7f97d9015640 1 -- 172.21.15.37:0/2114704734 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c402ae20 con 0x7f97d415fc10 2024-04-26T20:24:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.204+0000 7f97d9015640 1 -- 172.21.15.37:0/2114704734 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4032660 con 0x7f97d415fc10 2024-04-26T20:24:13.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.205+0000 7f97dc27b640 1 -- 172.21.15.37:0/2114704734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d415fc10 msgr2=0x7f97d41600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.205+0000 7f97dc27b640 1 --2- 172.21.15.37:0/2114704734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d415fc10 0x7f97d41600c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f97d4074710 tx=0x7f97c402a110 comp rx=0 tx=0).stop 2024-04-26T20:24:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.205+0000 7f97dc27b640 1 -- 172.21.15.37:0/2114704734 shutdown_connections 2024-04-26T20:24:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.205+0000 7f97dc27b640 1 --2- 172.21.15.37:0/2114704734 >> v2:172.21.15.134:3300/0 conn(0x7f97d40ce9b0 0x7f97d40cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.205+0000 7f97dc27b640 1 --2- 172.21.15.37:0/2114704734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d415fc10 0x7f97d41600c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.205+0000 7f97dc27b640 1 -- 172.21.15.37:0/2114704734 >> 172.21.15.37:0/2114704734 conn(0x7f97d40d0d50 msgr2=0x7f97d40d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.205+0000 7f97dc27b640 1 -- 172.21.15.37:0/2114704734 shutdown_connections 2024-04-26T20:24:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.205+0000 7f97dc27b640 1 -- 172.21.15.37:0/2114704734 wait complete. 2024-04-26T20:24:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.206+0000 7f97dc27b640 1 Processor -- start 2024-04-26T20:24:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.206+0000 7f97dc27b640 1 -- start start 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97dc27b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d40ce9b0 0x7f97d4152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97dc27b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97d4159a90 0x7f97d415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97dc27b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d415fc10 0x7f97d4145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97dc27b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97d4159e70 con 0x7f97d40ce9b0 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97dc27b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f97d4070300 con 0x7f97d415fc10 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97dc27b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f97d4159ff0 con 0x7f97d4159a90 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97da017640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d40ce9b0 0x7f97d4152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97da017640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d40ce9b0 0x7f97d4152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43884/0 (socket says 172.21.15.37:43884) 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97da017640 1 -- 172.21.15.37:0/3371062912 learned_addr learned my addr 172.21.15.37:0/3371062912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97da818640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d415fc10 0x7f97d4145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97d9816640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97d4159a90 0x7f97d415a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97da017640 1 -- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97d4159a90 msgr2=0x7f97d415a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97da017640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97d4159a90 0x7f97d415a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97da017640 1 -- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d415fc10 msgr2=0x7f97d4145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97da017640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d415fc10 0x7f97d4145570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97da017640 1 -- 172.21.15.37:0/3371062912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d4152c60 con 0x7f97d40ce9b0 2024-04-26T20:24:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97da818640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d415fc10 0x7f97d4145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.207+0000 7f97da017640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d40ce9b0 0x7f97d4152e80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c430 tx=0x7f97d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.208+0000 7f97caffd640 1 -- 172.21.15.37:0/3371062912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0011790 con 0x7f97d40ce9b0 2024-04-26T20:24:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.208+0000 7f97caffd640 1 -- 172.21.15.37:0/3371062912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d0017070 con 0x7f97d40ce9b0 2024-04-26T20:24:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.208+0000 7f97caffd640 1 -- 172.21.15.37:0/3371062912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0011bb0 con 0x7f97d40ce9b0 2024-04-26T20:24:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.208+0000 7f97dc27b640 1 -- 172.21.15.37:0/3371062912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d415a860 con 0x7f97d40ce9b0 2024-04-26T20:24:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.208+0000 7f97dc27b640 1 -- 172.21.15.37:0/3371062912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d415ae20 con 0x7f97d40ce9b0 2024-04-26T20:24:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.209+0000 7f97dc27b640 1 -- 172.21.15.37:0/3371062912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d406a360 con 0x7f97d40ce9b0 2024-04-26T20:24:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.209+0000 7f97caffd640 1 -- 172.21.15.37:0/3371062912 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f97d001c460 con 0x7f97d40ce9b0 2024-04-26T20:24:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.209+0000 7f97caffd640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97b806cbc0 0x7f97b806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.210+0000 7f97caffd640 1 -- 172.21.15.37:0/3371062912 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f97d008bec0 con 0x7f97d40ce9b0 2024-04-26T20:24:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.210+0000 7f97d9816640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97b806cbc0 0x7f97b806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.210+0000 7f97d9816640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97b806cbc0 0x7f97b806f080 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f97c4034000 tx=0x7f97c402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.214+0000 7f97caffd640 1 -- 172.21.15.37:0/3371062912 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f97d4152c60 con 0x7f97d40ce9b0 2024-04-26T20:24:13.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.398+0000 7f97dc27b640 1 -- 172.21.15.37:0/3371062912 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97d40d0730 con 0x7f97b806cbc0 2024-04-26T20:24:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.405+0000 7f97caffd640 1 -- 172.21.15.37:0/3371062912 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251956 (secure 0 0 0) 0x7f97d40d0730 con 0x7f97b806cbc0 2024-04-26T20:24:13.409 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:13.410 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:24:13.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.407+0000 7f97dc27b640 1 -- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97b806cbc0 msgr2=0x7f97b806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.407+0000 7f97dc27b640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97b806cbc0 0x7f97b806f080 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f97c4034000 tx=0x7f97c402e000 comp rx=0 tx=0).stop 2024-04-26T20:24:13.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.407+0000 7f97dc27b640 1 -- 172.21.15.37:0/3371062912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d40ce9b0 msgr2=0x7f97d4152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.407+0000 7f97dc27b640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d40ce9b0 0x7f97d4152e80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c430 tx=0x7f97d000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.407+0000 7f97dc27b640 1 -- 172.21.15.37:0/3371062912 shutdown_connections 2024-04-26T20:24:13.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.408+0000 7f97dc27b640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d415fc10 0x7f97d4145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.408+0000 7f97dc27b640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97b806cbc0 0x7f97b806f080 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.408+0000 7f97dc27b640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97d4159a90 0x7f97d415a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.408+0000 7f97dc27b640 1 --2- 172.21.15.37:0/3371062912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d40ce9b0 0x7f97d4152e80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.408+0000 7f97dc27b640 1 -- 172.21.15.37:0/3371062912 >> 172.21.15.37:0/3371062912 conn(0x7f97d40d0d50 msgr2=0x7f97d40d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:13.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.408+0000 7f97dc27b640 1 -- 172.21.15.37:0/3371062912 shutdown_connections 2024-04-26T20:24:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.408+0000 7f97dc27b640 1 -- 172.21.15.37:0/3371062912 wait complete. 2024-04-26T20:24:13.420 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-04-26T20:24:13.368858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234068,"kb_used_data":5400,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754740652,"statfs":{"total":773094113280,"available":772854427648,"internally_reserved":0,"allocated":5529600,"data_stored":2937612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.985389"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538141+0000","last_change":"2024-04-26T20:24:10.412044+0000","last_active":"2024-04-26T20:24:11.538141+0000","last_peered":"2024-04-26T20:24:11.538141+0000","last_clean":"2024-04-26T20:24:11.538141+0000","last_became_active":"2024-04-26T20:24:10.411672+0000","last_became_peered":"2024-04-26T20:24:10.411672+0000","last_unstale":"2024-04-26T20:24:11.538141+0000","last_undegraded":"2024-04-26T20:24:11.538141+0000","last_fullsized":"2024-04-26T20:24:11.538141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:14.142583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413075+0000","last_change":"2024-04-26T20:24:10.420421+0000","last_active":"2024-04-26T20:24:11.413075+0000","last_peered":"2024-04-26T20:24:11.413075+0000","last_clean":"2024-04-26T20:24:11.413075+0000","last_became_active":"2024-04-26T20:24:10.419930+0000","last_became_peered":"2024-04-26T20:24:10.419930+0000","last_unstale":"2024-04-26T20:24:11.413075+0000","last_undegraded":"2024-04-26T20:24:11.413075+0000","last_fullsized":"2024-04-26T20:24:11.413075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:35.477784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039673+0000","last_change":"2024-04-26T20:24:10.420914+0000","last_active":"2024-04-26T20:24:12.039673+0000","last_peered":"2024-04-26T20:24:12.039673+0000","last_clean":"2024-04-26T20:24:12.039673+0000","last_became_active":"2024-04-26T20:24:10.420040+0000","last_became_peered":"2024-04-26T20:24:10.420040+0000","last_unstale":"2024-04-26T20:24:12.039673+0000","last_undegraded":"2024-04-26T20:24:12.039673+0000","last_fullsized":"2024-04-26T20:24:12.039673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:26:57.481629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412336+0000","last_change":"2024-04-26T20:24:10.410426+0000","last_active":"2024-04-26T20:24:11.412336+0000","last_peered":"2024-04-26T20:24:11.412336+0000","last_clean":"2024-04-26T20:24:11.412336+0000","last_became_active":"2024-04-26T20:24:10.410115+0000","last_became_peered":"2024-04-26T20:24:10.410115+0000","last_unstale":"2024-04-26T20:24:11.412336+0000","last_undegraded":"2024-04-26T20:24:11.412336+0000","last_fullsized":"2024-04-26T20:24:11.412336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:26:57.481629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412804+0000","last_change":"2024-04-26T20:24:10.411822+0000","last_active":"2024-04-26T20:24:11.412804+0000","last_peered":"2024-04-26T20:24:11.412804+0000","last_clean":"2024-04-26T20:24:11.412804+0000","last_became_active":"2024-04-26T20:24:10.411514+0000","last_became_peered":"2024-04-26T20:24:10.411514+0000","last_unstale":"2024-04-26T20:24:11.412804+0000","last_undegraded":"2024-04-26T20:24:11.412804+0000","last_fullsized":"2024-04-26T20:24:11.412804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:41.710250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413181+0000","last_change":"2024-04-26T20:24:10.414904+0000","last_active":"2024-04-26T20:24:11.413181+0000","last_peered":"2024-04-26T20:24:11.413181+0000","last_clean":"2024-04-26T20:24:11.413181+0000","last_became_active":"2024-04-26T20:24:10.414625+0000","last_became_peered":"2024-04-26T20:24:10.414625+0000","last_unstale":"2024-04-26T20:24:11.413181+0000","last_undegraded":"2024-04-26T20:24:11.413181+0000","last_fullsized":"2024-04-26T20:24:11.413181+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:14.142583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538012+0000","last_change":"2024-04-26T20:24:10.420989+0000","last_active":"2024-04-26T20:24:11.538012+0000","last_peered":"2024-04-26T20:24:11.538012+0000","last_clean":"2024-04-26T20:24:11.538012+0000","last_became_active":"2024-04-26T20:24:10.420621+0000","last_became_peered":"2024-04-26T20:24:10.420621+0000","last_unstale":"2024-04-26T20:24:11.538012+0000","last_undegraded":"2024-04-26T20:24:11.538012+0000","last_fullsized":"2024-04-26T20:24:11.538012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:48.412809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412007+0000","last_change":"2024-04-26T20:24:10.411191+0000","last_active":"2024-04-26T20:24:11.412007+0000","last_peered":"2024-04-26T20:24:11.412007+0000","last_clean":"2024-04-26T20:24:11.412007+0000","last_became_active":"2024-04-26T20:24:10.410886+0000","last_became_peered":"2024-04-26T20:24:10.410886+0000","last_unstale":"2024-04-26T20:24:11.412007+0000","last_undegraded":"2024-04-26T20:24:11.412007+0000","last_fullsized":"2024-04-26T20:24:11.412007+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:44.412522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412286+0000","last_change":"2024-04-26T20:24:10.418832+0000","last_active":"2024-04-26T20:24:11.412286+0000","last_peered":"2024-04-26T20:24:11.412286+0000","last_clean":"2024-04-26T20:24:11.412286+0000","last_became_active":"2024-04-26T20:24:10.418549+0000","last_became_peered":"2024-04-26T20:24:10.418549+0000","last_unstale":"2024-04-26T20:24:11.412286+0000","last_undegraded":"2024-04-26T20:24:11.412286+0000","last_fullsized":"2024-04-26T20:24:11.412286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:36.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412598+0000","last_change":"2024-04-26T20:24:10.416343+0000","last_active":"2024-04-26T20:24:11.412598+0000","last_peered":"2024-04-26T20:24:11.412598+0000","last_clean":"2024-04-26T20:24:11.412598+0000","last_became_active":"2024-04-26T20:24:10.415517+0000","last_became_peered":"2024-04-26T20:24:10.415517+0000","last_unstale":"2024-04-26T20:24:11.412598+0000","last_undegraded":"2024-04-26T20:24:11.412598+0000","last_fullsized":"2024-04-26T20:24:11.412598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:41.710250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.411991+0000","last_change":"2024-04-26T20:24:10.419554+0000","last_active":"2024-04-26T20:24:11.411991+0000","last_peered":"2024-04-26T20:24:11.411991+0000","last_clean":"2024-04-26T20:24:11.411991+0000","last_became_active":"2024-04-26T20:24:10.419263+0000","last_became_peered":"2024-04-26T20:24:10.419263+0000","last_unstale":"2024-04-26T20:24:11.411991+0000","last_undegraded":"2024-04-26T20:24:11.411991+0000","last_fullsized":"2024-04-26T20:24:11.411991+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:31.589088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412924+0000","last_change":"2024-04-26T20:24:10.416431+0000","last_active":"2024-04-26T20:24:11.412924+0000","last_peered":"2024-04-26T20:24:11.412924+0000","last_clean":"2024-04-26T20:24:11.412924+0000","last_became_active":"2024-04-26T20:24:10.416133+0000","last_became_peered":"2024-04-26T20:24:10.416133+0000","last_unstale":"2024-04-26T20:24:11.412924+0000","last_undegraded":"2024-04-26T20:24:11.412924+0000","last_fullsized":"2024-04-26T20:24:11.412924+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:36.412020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039573+0000","last_change":"2024-04-26T20:24:10.421697+0000","last_active":"2024-04-26T20:24:12.039573+0000","last_peered":"2024-04-26T20:24:12.039573+0000","last_clean":"2024-04-26T20:24:12.039573+0000","last_became_active":"2024-04-26T20:24:10.421262+0000","last_became_peered":"2024-04-26T20:24:10.421262+0000","last_unstale":"2024-04-26T20:24:12.039573+0000","last_undegraded":"2024-04-26T20:24:12.039573+0000","last_fullsized":"2024-04-26T20:24:12.039573+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:46.762865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412079+0000","last_change":"2024-04-26T20:24:10.418165+0000","last_active":"2024-04-26T20:24:11.412079+0000","last_peered":"2024-04-26T20:24:11.412079+0000","last_clean":"2024-04-26T20:24:11.412079+0000","last_became_active":"2024-04-26T20:24:10.417893+0000","last_became_peered":"2024-04-26T20:24:10.417893+0000","last_unstale":"2024-04-26T20:24:11.412079+0000","last_undegraded":"2024-04-26T20:24:11.412079+0000","last_fullsized":"2024-04-26T20:24:11.412079+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:06.755611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413083+0000","last_change":"2024-04-26T20:24:10.409013+0000","last_active":"2024-04-26T20:24:11.413083+0000","last_peered":"2024-04-26T20:24:11.413083+0000","last_clean":"2024-04-26T20:24:11.413083+0000","last_became_active":"2024-04-26T20:24:10.408383+0000","last_became_peered":"2024-04-26T20:24:10.408383+0000","last_unstale":"2024-04-26T20:24:11.413083+0000","last_undegraded":"2024-04-26T20:24:11.413083+0000","last_fullsized":"2024-04-26T20:24:11.413083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:29.253761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039694+0000","last_change":"2024-04-26T20:24:10.418862+0000","last_active":"2024-04-26T20:24:12.039694+0000","last_peered":"2024-04-26T20:24:12.039694+0000","last_clean":"2024-04-26T20:24:12.039694+0000","last_became_active":"2024-04-26T20:24:10.418589+0000","last_became_peered":"2024-04-26T20:24:10.418589+0000","last_unstale":"2024-04-26T20:24:12.039694+0000","last_undegraded":"2024-04-26T20:24:12.039694+0000","last_fullsized":"2024-04-26T20:24:12.039694+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:16.533192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040093+0000","last_change":"2024-04-26T20:24:10.415295+0000","last_active":"2024-04-26T20:24:12.040093+0000","last_peered":"2024-04-26T20:24:12.040093+0000","last_clean":"2024-04-26T20:24:12.040093+0000","last_became_active":"2024-04-26T20:24:10.415005+0000","last_became_peered":"2024-04-26T20:24:10.415005+0000","last_unstale":"2024-04-26T20:24:12.040093+0000","last_undegraded":"2024-04-26T20:24:12.040093+0000","last_fullsized":"2024-04-26T20:24:12.040093+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:59.031860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040298+0000","last_change":"2024-04-26T20:24:10.421863+0000","last_active":"2024-04-26T20:24:12.040298+0000","last_peered":"2024-04-26T20:24:12.040298+0000","last_clean":"2024-04-26T20:24:12.040298+0000","last_became_active":"2024-04-26T20:24:10.421570+0000","last_became_peered":"2024-04-26T20:24:10.421570+0000","last_unstale":"2024-04-26T20:24:12.040298+0000","last_undegraded":"2024-04-26T20:24:12.040298+0000","last_fullsized":"2024-04-26T20:24:12.040298+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:09.962717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040532+0000","last_change":"2024-04-26T20:24:10.425654+0000","last_active":"2024-04-26T20:24:12.040532+0000","last_peered":"2024-04-26T20:24:12.040532+0000","last_clean":"2024-04-26T20:24:12.040532+0000","last_became_active":"2024-04-26T20:24:10.425418+0000","last_became_peered":"2024-04-26T20:24:10.425418+0000","last_unstale":"2024-04-26T20:24:12.040532+0000","last_undegraded":"2024-04-26T20:24:12.040532+0000","last_fullsized":"2024-04-26T20:24:12.040532+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:47.190265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039790+0000","last_change":"2024-04-26T20:24:10.425637+0000","last_active":"2024-04-26T20:24:12.039790+0000","last_peered":"2024-04-26T20:24:12.039790+0000","last_clean":"2024-04-26T20:24:12.039790+0000","last_became_active":"2024-04-26T20:24:10.425294+0000","last_became_peered":"2024-04-26T20:24:10.425294+0000","last_unstale":"2024-04-26T20:24:12.039790+0000","last_undegraded":"2024-04-26T20:24:12.039790+0000","last_fullsized":"2024-04-26T20:24:12.039790+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:03.150219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413190+0000","last_change":"2024-04-26T20:24:10.416227+0000","last_active":"2024-04-26T20:24:11.413190+0000","last_peered":"2024-04-26T20:24:11.413190+0000","last_clean":"2024-04-26T20:24:11.413190+0000","last_became_active":"2024-04-26T20:24:10.415783+0000","last_became_peered":"2024-04-26T20:24:10.415783+0000","last_unstale":"2024-04-26T20:24:11.413190+0000","last_undegraded":"2024-04-26T20:24:11.413190+0000","last_fullsized":"2024-04-26T20:24:11.413190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:53.099897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412388+0000","last_change":"2024-04-26T20:24:10.412142+0000","last_active":"2024-04-26T20:24:11.412388+0000","last_peered":"2024-04-26T20:24:11.412388+0000","last_clean":"2024-04-26T20:24:11.412388+0000","last_became_active":"2024-04-26T20:24:10.411703+0000","last_became_peered":"2024-04-26T20:24:10.411703+0000","last_unstale":"2024-04-26T20:24:11.412388+0000","last_undegraded":"2024-04-26T20:24:11.412388+0000","last_fullsized":"2024-04-26T20:24:11.412388+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:21.440553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040288+0000","last_change":"2024-04-26T20:24:10.412984+0000","last_active":"2024-04-26T20:24:12.040288+0000","last_peered":"2024-04-26T20:24:12.040288+0000","last_clean":"2024-04-26T20:24:12.040288+0000","last_became_active":"2024-04-26T20:24:10.412668+0000","last_became_peered":"2024-04-26T20:24:10.412668+0000","last_unstale":"2024-04-26T20:24:12.040288+0000","last_undegraded":"2024-04-26T20:24:12.040288+0000","last_fullsized":"2024-04-26T20:24:12.040288+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:30.101850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039884+0000","last_change":"2024-04-26T20:24:10.413134+0000","last_active":"2024-04-26T20:24:12.039884+0000","last_peered":"2024-04-26T20:24:12.039884+0000","last_clean":"2024-04-26T20:24:12.039884+0000","last_became_active":"2024-04-26T20:24:10.412804+0000","last_became_peered":"2024-04-26T20:24:10.412804+0000","last_unstale":"2024-04-26T20:24:12.039884+0000","last_undegraded":"2024-04-26T20:24:12.039884+0000","last_fullsized":"2024-04-26T20:24:12.039884+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:03.150219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040559+0000","last_change":"2024-04-26T20:24:10.411413+0000","last_active":"2024-04-26T20:24:12.040559+0000","last_peered":"2024-04-26T20:24:12.040559+0000","last_clean":"2024-04-26T20:24:12.040559+0000","last_became_active":"2024-04-26T20:24:10.411160+0000","last_became_peered":"2024-04-26T20:24:10.411160+0000","last_unstale":"2024-04-26T20:24:12.040559+0000","last_undegraded":"2024-04-26T20:24:12.040559+0000","last_fullsized":"2024-04-26T20:24:12.040559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:27.571285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039546+0000","last_change":"2024-04-26T20:24:10.420681+0000","last_active":"2024-04-26T20:24:12.039546+0000","last_peered":"2024-04-26T20:24:12.039546+0000","last_clean":"2024-04-26T20:24:12.039546+0000","last_became_active":"2024-04-26T20:24:10.420282+0000","last_became_peered":"2024-04-26T20:24:10.420282+0000","last_unstale":"2024-04-26T20:24:12.039546+0000","last_undegraded":"2024-04-26T20:24:12.039546+0000","last_fullsized":"2024-04-26T20:24:12.039546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:28.586657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039863+0000","last_change":"2024-04-26T20:24:10.411128+0000","last_active":"2024-04-26T20:24:12.039863+0000","last_peered":"2024-04-26T20:24:12.039863+0000","last_clean":"2024-04-26T20:24:12.039863+0000","last_became_active":"2024-04-26T20:24:10.410862+0000","last_became_peered":"2024-04-26T20:24:10.410862+0000","last_unstale":"2024-04-26T20:24:12.039863+0000","last_undegraded":"2024-04-26T20:24:12.039863+0000","last_fullsized":"2024-04-26T20:24:12.039863+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:21.440553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412990+0000","last_change":"2024-04-26T20:24:10.415403+0000","last_active":"2024-04-26T20:24:11.412990+0000","last_peered":"2024-04-26T20:24:11.412990+0000","last_clean":"2024-04-26T20:24:11.412990+0000","last_became_active":"2024-04-26T20:24:10.415147+0000","last_became_peered":"2024-04-26T20:24:10.415147+0000","last_unstale":"2024-04-26T20:24:11.412990+0000","last_undegraded":"2024-04-26T20:24:11.412990+0000","last_fullsized":"2024-04-26T20:24:11.412990+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:17.113224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040356+0000","last_change":"2024-04-26T20:24:10.410620+0000","last_active":"2024-04-26T20:24:12.040356+0000","last_peered":"2024-04-26T20:24:12.040356+0000","last_clean":"2024-04-26T20:24:12.040356+0000","last_became_active":"2024-04-26T20:24:10.410259+0000","last_became_peered":"2024-04-26T20:24:10.410259+0000","last_unstale":"2024-04-26T20:24:12.040356+0000","last_undegraded":"2024-04-26T20:24:12.040356+0000","last_fullsized":"2024-04-26T20:24:12.040356+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:35.010247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039526+0000","last_change":"2024-04-26T20:24:10.413050+0000","last_active":"2024-04-26T20:24:12.039526+0000","last_peered":"2024-04-26T20:24:12.039526+0000","last_clean":"2024-04-26T20:24:12.039526+0000","last_became_active":"2024-04-26T20:24:10.412454+0000","last_became_peered":"2024-04-26T20:24:10.412454+0000","last_unstale":"2024-04-26T20:24:12.039526+0000","last_undegraded":"2024-04-26T20:24:12.039526+0000","last_fullsized":"2024-04-26T20:24:12.039526+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:43.189695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413195+0000","last_change":"2024-04-26T20:24:10.409258+0000","last_active":"2024-04-26T20:24:11.413195+0000","last_peered":"2024-04-26T20:24:11.413195+0000","last_clean":"2024-04-26T20:24:11.413195+0000","last_became_active":"2024-04-26T20:24:10.408741+0000","last_became_peered":"2024-04-26T20:24:10.408741+0000","last_unstale":"2024-04-26T20:24:11.413195+0000","last_undegraded":"2024-04-26T20:24:11.413195+0000","last_fullsized":"2024-04-26T20:24:11.413195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:51.460143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039414+0000","last_change":"2024-04-26T20:24:10.416638+0000","last_active":"2024-04-26T20:24:12.039414+0000","last_peered":"2024-04-26T20:24:12.039414+0000","last_clean":"2024-04-26T20:24:12.039414+0000","last_became_active":"2024-04-26T20:24:10.415734+0000","last_became_peered":"2024-04-26T20:24:10.415734+0000","last_unstale":"2024-04-26T20:24:12.039414+0000","last_undegraded":"2024-04-26T20:24:12.039414+0000","last_fullsized":"2024-04-26T20:24:12.039414+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:10.932936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611360+0000","last_change":"2024-04-26T20:24:10.416676+0000","last_active":"2024-04-26T20:24:11.611360+0000","last_peered":"2024-04-26T20:24:11.611360+0000","last_clean":"2024-04-26T20:24:11.611360+0000","last_became_active":"2024-04-26T20:24:10.416413+0000","last_became_peered":"2024-04-26T20:24:10.416413+0000","last_unstale":"2024-04-26T20:24:11.611360+0000","last_undegraded":"2024-04-26T20:24:11.611360+0000","last_fullsized":"2024-04-26T20:24:11.611360+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:31.589088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413644+0000","last_change":"2024-04-26T20:24:09.400089+0000","last_active":"2024-04-26T20:24:11.413644+0000","last_peered":"2024-04-26T20:24:11.413644+0000","last_clean":"2024-04-26T20:24:11.413644+0000","last_became_active":"2024-04-26T20:24:09.399799+0000","last_became_peered":"2024-04-26T20:24:09.399799+0000","last_unstale":"2024-04-26T20:24:11.413644+0000","last_undegraded":"2024-04-26T20:24:11.413644+0000","last_fullsized":"2024-04-26T20:24:11.413644+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:34.472483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040692+0000","last_change":"2024-04-26T20:24:10.407406+0000","last_active":"2024-04-26T20:24:12.040692+0000","last_peered":"2024-04-26T20:24:12.040692+0000","last_clean":"2024-04-26T20:24:12.040692+0000","last_became_active":"2024-04-26T20:24:10.407106+0000","last_became_peered":"2024-04-26T20:24:10.407106+0000","last_unstale":"2024-04-26T20:24:12.040692+0000","last_undegraded":"2024-04-26T20:24:12.040692+0000","last_fullsized":"2024-04-26T20:24:12.040692+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:34.931765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040070+0000","last_change":"2024-04-26T20:24:09.402566+0000","last_active":"2024-04-26T20:24:12.040070+0000","last_peered":"2024-04-26T20:24:12.040070+0000","last_clean":"2024-04-26T20:24:12.040070+0000","last_became_active":"2024-04-26T20:24:09.402281+0000","last_became_peered":"2024-04-26T20:24:09.402281+0000","last_unstale":"2024-04-26T20:24:12.040070+0000","last_undegraded":"2024-04-26T20:24:12.040070+0000","last_fullsized":"2024-04-26T20:24:12.040070+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:20.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413182+0000","last_change":"2024-04-26T20:24:10.418192+0000","last_active":"2024-04-26T20:24:11.413182+0000","last_peered":"2024-04-26T20:24:11.413182+0000","last_clean":"2024-04-26T20:24:11.413182+0000","last_became_active":"2024-04-26T20:24:10.417863+0000","last_became_peered":"2024-04-26T20:24:10.417863+0000","last_unstale":"2024-04-26T20:24:11.413182+0000","last_undegraded":"2024-04-26T20:24:11.413182+0000","last_fullsized":"2024-04-26T20:24:11.413182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:45.455882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611441+0000","last_change":"2024-04-26T20:24:09.410058+0000","last_active":"2024-04-26T20:24:11.611441+0000","last_peered":"2024-04-26T20:24:11.611441+0000","last_clean":"2024-04-26T20:24:11.611441+0000","last_became_active":"2024-04-26T20:24:09.409706+0000","last_became_peered":"2024-04-26T20:24:09.409706+0000","last_unstale":"2024-04-26T20:24:11.611441+0000","last_undegraded":"2024-04-26T20:24:11.611441+0000","last_fullsized":"2024-04-26T20:24:11.611441+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:33.926464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611493+0000","last_change":"2024-04-26T20:24:10.410080+0000","last_active":"2024-04-26T20:24:11.611493+0000","last_peered":"2024-04-26T20:24:11.611493+0000","last_clean":"2024-04-26T20:24:11.611493+0000","last_became_active":"2024-04-26T20:24:10.409736+0000","last_became_peered":"2024-04-26T20:24:10.409736+0000","last_unstale":"2024-04-26T20:24:11.611493+0000","last_undegraded":"2024-04-26T20:24:11.611493+0000","last_fullsized":"2024-04-26T20:24:11.611493+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:44.412522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.537922+0000","last_change":"2024-04-26T20:24:09.405279+0000","last_active":"2024-04-26T20:24:11.537922+0000","last_peered":"2024-04-26T20:24:11.537922+0000","last_clean":"2024-04-26T20:24:11.537922+0000","last_became_active":"2024-04-26T20:24:09.404980+0000","last_became_peered":"2024-04-26T20:24:09.404980+0000","last_unstale":"2024-04-26T20:24:11.537922+0000","last_undegraded":"2024-04-26T20:24:11.537922+0000","last_fullsized":"2024-04-26T20:24:11.537922+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:44.450580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039493+0000","last_change":"2024-04-26T20:24:10.409268+0000","last_active":"2024-04-26T20:24:12.039493+0000","last_peered":"2024-04-26T20:24:12.039493+0000","last_clean":"2024-04-26T20:24:12.039493+0000","last_became_active":"2024-04-26T20:24:10.408997+0000","last_became_peered":"2024-04-26T20:24:10.408997+0000","last_unstale":"2024-04-26T20:24:12.039493+0000","last_undegraded":"2024-04-26T20:24:12.039493+0000","last_fullsized":"2024-04-26T20:24:12.039493+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:06.755611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413485+0000","last_change":"2024-04-26T20:24:09.405281+0000","last_active":"2024-04-26T20:24:11.413485+0000","last_peered":"2024-04-26T20:24:11.413485+0000","last_clean":"2024-04-26T20:24:11.413485+0000","last_became_active":"2024-04-26T20:24:09.404954+0000","last_became_peered":"2024-04-26T20:24:09.404954+0000","last_unstale":"2024-04-26T20:24:11.413485+0000","last_undegraded":"2024-04-26T20:24:11.413485+0000","last_fullsized":"2024-04-26T20:24:11.413485+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:50.422477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412148+0000","last_change":"2024-04-26T20:24:10.425832+0000","last_active":"2024-04-26T20:24:11.412148+0000","last_peered":"2024-04-26T20:24:11.412148+0000","last_clean":"2024-04-26T20:24:11.412148+0000","last_became_active":"2024-04-26T20:24:10.425361+0000","last_became_peered":"2024-04-26T20:24:10.425361+0000","last_unstale":"2024-04-26T20:24:11.412148+0000","last_undegraded":"2024-04-26T20:24:11.412148+0000","last_fullsized":"2024-04-26T20:24:11.412148+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:43.189695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040300+0000","last_change":"2024-04-26T20:24:09.408539+0000","last_active":"2024-04-26T20:24:12.040300+0000","last_peered":"2024-04-26T20:24:12.040300+0000","last_clean":"2024-04-26T20:24:12.040300+0000","last_became_active":"2024-04-26T20:24:09.408022+0000","last_became_peered":"2024-04-26T20:24:09.408022+0000","last_unstale":"2024-04-26T20:24:12.040300+0000","last_undegraded":"2024-04-26T20:24:12.040300+0000","last_fullsized":"2024-04-26T20:24:12.040300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:34.004945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.411807+0000","last_change":"2024-04-26T20:24:10.419051+0000","last_active":"2024-04-26T20:24:11.411807+0000","last_peered":"2024-04-26T20:24:11.411807+0000","last_clean":"2024-04-26T20:24:11.411807+0000","last_became_active":"2024-04-26T20:24:10.412887+0000","last_became_peered":"2024-04-26T20:24:10.412887+0000","last_unstale":"2024-04-26T20:24:11.411807+0000","last_undegraded":"2024-04-26T20:24:11.411807+0000","last_fullsized":"2024-04-26T20:24:11.411807+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:28.821951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412653+0000","last_change":"2024-04-26T20:24:09.407265+0000","last_active":"2024-04-26T20:24:11.412653+0000","last_peered":"2024-04-26T20:24:11.412653+0000","last_clean":"2024-04-26T20:24:11.412653+0000","last_became_active":"2024-04-26T20:24:09.406440+0000","last_became_peered":"2024-04-26T20:24:09.406440+0000","last_unstale":"2024-04-26T20:24:11.412653+0000","last_undegraded":"2024-04-26T20:24:11.412653+0000","last_fullsized":"2024-04-26T20:24:11.412653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:30.583786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040142+0000","last_change":"2024-04-26T20:24:10.412988+0000","last_active":"2024-04-26T20:24:12.040142+0000","last_peered":"2024-04-26T20:24:12.040142+0000","last_clean":"2024-04-26T20:24:12.040142+0000","last_became_active":"2024-04-26T20:24:10.412605+0000","last_became_peered":"2024-04-26T20:24:10.412605+0000","last_unstale":"2024-04-26T20:24:12.040142+0000","last_undegraded":"2024-04-26T20:24:12.040142+0000","last_fullsized":"2024-04-26T20:24:12.040142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:54.004667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412365+0000","last_change":"2024-04-26T20:24:09.404738+0000","last_active":"2024-04-26T20:24:11.412365+0000","last_peered":"2024-04-26T20:24:11.412365+0000","last_clean":"2024-04-26T20:24:11.412365+0000","last_became_active":"2024-04-26T20:24:09.404407+0000","last_became_peered":"2024-04-26T20:24:09.404407+0000","last_unstale":"2024-04-26T20:24:11.412365+0000","last_undegraded":"2024-04-26T20:24:11.412365+0000","last_fullsized":"2024-04-26T20:24:11.412365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:27.581356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039877+0000","last_change":"2024-04-26T20:24:10.421571+0000","last_active":"2024-04-26T20:24:12.039877+0000","last_peered":"2024-04-26T20:24:12.039877+0000","last_clean":"2024-04-26T20:24:12.039877+0000","last_became_active":"2024-04-26T20:24:10.421406+0000","last_became_peered":"2024-04-26T20:24:10.421406+0000","last_unstale":"2024-04-26T20:24:12.039877+0000","last_undegraded":"2024-04-26T20:24:12.039877+0000","last_fullsized":"2024-04-26T20:24:12.039877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:22.506318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040280+0000","last_change":"2024-04-26T20:24:09.408668+0000","last_active":"2024-04-26T20:24:12.040280+0000","last_peered":"2024-04-26T20:24:12.040280+0000","last_clean":"2024-04-26T20:24:12.040280+0000","last_became_active":"2024-04-26T20:24:09.408423+0000","last_became_peered":"2024-04-26T20:24:09.408423+0000","last_unstale":"2024-04-26T20:24:12.040280+0000","last_undegraded":"2024-04-26T20:24:12.040280+0000","last_fullsized":"2024-04-26T20:24:12.040280+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:26.565983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538065+0000","last_change":"2024-04-26T20:24:10.419074+0000","last_active":"2024-04-26T20:24:11.538065+0000","last_peered":"2024-04-26T20:24:11.538065+0000","last_clean":"2024-04-26T20:24:11.538065+0000","last_became_active":"2024-04-26T20:24:10.418701+0000","last_became_peered":"2024-04-26T20:24:10.418701+0000","last_unstale":"2024-04-26T20:24:11.538065+0000","last_undegraded":"2024-04-26T20:24:11.538065+0000","last_fullsized":"2024-04-26T20:24:11.538065+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:29.253761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412680+0000","last_change":"2024-04-26T20:24:09.409147+0000","last_active":"2024-04-26T20:24:11.412680+0000","last_peered":"2024-04-26T20:24:11.412680+0000","last_clean":"2024-04-26T20:24:11.412680+0000","last_became_active":"2024-04-26T20:24:09.408634+0000","last_became_peered":"2024-04-26T20:24:09.408634+0000","last_unstale":"2024-04-26T20:24:11.412680+0000","last_undegraded":"2024-04-26T20:24:11.412680+0000","last_fullsized":"2024-04-26T20:24:11.412680+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:17.421286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413428+0000","last_change":"2024-04-26T20:24:10.414946+0000","last_active":"2024-04-26T20:24:11.413428+0000","last_peered":"2024-04-26T20:24:11.413428+0000","last_clean":"2024-04-26T20:24:11.413428+0000","last_became_active":"2024-04-26T20:24:10.414630+0000","last_became_peered":"2024-04-26T20:24:10.414630+0000","last_unstale":"2024-04-26T20:24:11.413428+0000","last_undegraded":"2024-04-26T20:24:11.413428+0000","last_fullsized":"2024-04-26T20:24:11.413428+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:25.267112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538154+0000","last_change":"2024-04-26T20:24:09.408914+0000","last_active":"2024-04-26T20:24:11.538154+0000","last_peered":"2024-04-26T20:24:11.538154+0000","last_clean":"2024-04-26T20:24:11.538154+0000","last_became_active":"2024-04-26T20:24:09.408610+0000","last_became_peered":"2024-04-26T20:24:09.408610+0000","last_unstale":"2024-04-26T20:24:11.538154+0000","last_undegraded":"2024-04-26T20:24:11.538154+0000","last_fullsized":"2024-04-26T20:24:11.538154+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:06.566389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538022+0000","last_change":"2024-04-26T20:24:10.420829+0000","last_active":"2024-04-26T20:24:11.538022+0000","last_peered":"2024-04-26T20:24:11.538022+0000","last_clean":"2024-04-26T20:24:11.538022+0000","last_became_active":"2024-04-26T20:24:10.420568+0000","last_became_peered":"2024-04-26T20:24:10.420568+0000","last_unstale":"2024-04-26T20:24:11.538022+0000","last_undegraded":"2024-04-26T20:24:11.538022+0000","last_fullsized":"2024-04-26T20:24:11.538022+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:33.660671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039773+0000","last_change":"2024-04-26T20:24:09.409891+0000","last_active":"2024-04-26T20:24:12.039773+0000","last_peered":"2024-04-26T20:24:12.039773+0000","last_clean":"2024-04-26T20:24:12.039773+0000","last_became_active":"2024-04-26T20:24:09.409604+0000","last_became_peered":"2024-04-26T20:24:09.409604+0000","last_unstale":"2024-04-26T20:24:12.039773+0000","last_undegraded":"2024-04-26T20:24:12.039773+0000","last_fullsized":"2024-04-26T20:24:12.039773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:17.677447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040028+0000","last_change":"2024-04-26T20:24:10.425603+0000","last_active":"2024-04-26T20:24:12.040028+0000","last_peered":"2024-04-26T20:24:12.040028+0000","last_clean":"2024-04-26T20:24:12.040028+0000","last_became_active":"2024-04-26T20:24:10.425317+0000","last_became_peered":"2024-04-26T20:24:10.425317+0000","last_unstale":"2024-04-26T20:24:12.040028+0000","last_undegraded":"2024-04-26T20:24:12.040028+0000","last_fullsized":"2024-04-26T20:24:12.040028+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:43.697260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413479+0000","last_change":"2024-04-26T20:24:09.408958+0000","last_active":"2024-04-26T20:24:11.413479+0000","last_peered":"2024-04-26T20:24:11.413479+0000","last_clean":"2024-04-26T20:24:11.413479+0000","last_became_active":"2024-04-26T20:24:09.408684+0000","last_became_peered":"2024-04-26T20:24:09.408684+0000","last_unstale":"2024-04-26T20:24:11.413479+0000","last_undegraded":"2024-04-26T20:24:11.413479+0000","last_fullsized":"2024-04-26T20:24:11.413479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:44.450580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412523+0000","last_change":"2024-04-26T20:24:10.412257+0000","last_active":"2024-04-26T20:24:11.412523+0000","last_peered":"2024-04-26T20:24:11.412523+0000","last_clean":"2024-04-26T20:24:11.412523+0000","last_became_active":"2024-04-26T20:24:10.411429+0000","last_became_peered":"2024-04-26T20:24:10.411429+0000","last_unstale":"2024-04-26T20:24:11.412523+0000","last_undegraded":"2024-04-26T20:24:11.412523+0000","last_fullsized":"2024-04-26T20:24:11.412523+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:59.031860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538194+0000","last_change":"2024-04-26T20:24:09.406698+0000","last_active":"2024-04-26T20:24:11.538194+0000","last_peered":"2024-04-26T20:24:11.538194+0000","last_clean":"2024-04-26T20:24:11.538194+0000","last_became_active":"2024-04-26T20:24:09.406430+0000","last_became_peered":"2024-04-26T20:24:09.406430+0000","last_unstale":"2024-04-26T20:24:11.538194+0000","last_undegraded":"2024-04-26T20:24:11.538194+0000","last_fullsized":"2024-04-26T20:24:11.538194+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:16.107922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413041+0000","last_change":"2024-04-26T20:24:10.409367+0000","last_active":"2024-04-26T20:24:11.413041+0000","last_peered":"2024-04-26T20:24:11.413041+0000","last_clean":"2024-04-26T20:24:11.413041+0000","last_became_active":"2024-04-26T20:24:10.409060+0000","last_became_peered":"2024-04-26T20:24:10.409060+0000","last_unstale":"2024-04-26T20:24:11.413041+0000","last_undegraded":"2024-04-26T20:24:11.413041+0000","last_fullsized":"2024-04-26T20:24:11.413041+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:33.660671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413322+0000","last_change":"2024-04-26T20:24:09.408235+0000","last_active":"2024-04-26T20:24:11.413322+0000","last_peered":"2024-04-26T20:24:11.413322+0000","last_clean":"2024-04-26T20:24:11.413322+0000","last_became_active":"2024-04-26T20:24:09.407953+0000","last_became_peered":"2024-04-26T20:24:09.407953+0000","last_unstale":"2024-04-26T20:24:11.413322+0000","last_undegraded":"2024-04-26T20:24:11.413322+0000","last_fullsized":"2024-04-26T20:24:11.413322+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:16.107922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538183+0000","last_change":"2024-04-26T20:24:10.417833+0000","last_active":"2024-04-26T20:24:11.538183+0000","last_peered":"2024-04-26T20:24:11.538183+0000","last_clean":"2024-04-26T20:24:11.538183+0000","last_became_active":"2024-04-26T20:24:10.417473+0000","last_became_peered":"2024-04-26T20:24:10.417473+0000","last_unstale":"2024-04-26T20:24:11.538183+0000","last_undegraded":"2024-04-26T20:24:11.538183+0000","last_fullsized":"2024-04-26T20:24:11.538183+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:51.460143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":22,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412298+0000","last_change":"2024-04-26T20:24:09.410011+0000","last_active":"2024-04-26T20:24:11.412298+0000","last_peered":"2024-04-26T20:24:11.412298+0000","last_clean":"2024-04-26T20:24:11.412298+0000","last_became_active":"2024-04-26T20:24:09.409716+0000","last_became_peered":"2024-04-26T20:24:09.409716+0000","last_unstale":"2024-04-26T20:24:11.412298+0000","last_undegraded":"2024-04-26T20:24:11.412298+0000","last_fullsized":"2024-04-26T20:24:11.412298+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:15.527890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039598+0000","last_change":"2024-04-26T20:24:10.410349+0000","last_active":"2024-04-26T20:24:12.039598+0000","last_peered":"2024-04-26T20:24:12.039598+0000","last_clean":"2024-04-26T20:24:12.039598+0000","last_became_active":"2024-04-26T20:24:10.410033+0000","last_became_peered":"2024-04-26T20:24:10.410033+0000","last_unstale":"2024-04-26T20:24:12.039598+0000","last_undegraded":"2024-04-26T20:24:12.039598+0000","last_fullsized":"2024-04-26T20:24:12.039598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:16.533192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039623+0000","last_change":"2024-04-26T20:24:09.407042+0000","last_active":"2024-04-26T20:24:12.039623+0000","last_peered":"2024-04-26T20:24:12.039623+0000","last_clean":"2024-04-26T20:24:12.039623+0000","last_became_active":"2024-04-26T20:24:09.406397+0000","last_became_peered":"2024-04-26T20:24:09.406397+0000","last_unstale":"2024-04-26T20:24:12.039623+0000","last_undegraded":"2024-04-26T20:24:12.039623+0000","last_fullsized":"2024-04-26T20:24:12.039623+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:32.120531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412572+0000","last_change":"2024-04-26T20:24:10.422603+0000","last_active":"2024-04-26T20:24:11.412572+0000","last_peered":"2024-04-26T20:24:11.412572+0000","last_clean":"2024-04-26T20:24:11.412572+0000","last_became_active":"2024-04-26T20:24:10.422331+0000","last_became_peered":"2024-04-26T20:24:10.422331+0000","last_unstale":"2024-04-26T20:24:11.412572+0000","last_undegraded":"2024-04-26T20:24:11.412572+0000","last_fullsized":"2024-04-26T20:24:11.412572+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:09.962717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040181+0000","last_change":"2024-04-26T20:24:09.404863+0000","last_active":"2024-04-26T20:24:12.040181+0000","last_peered":"2024-04-26T20:24:12.040181+0000","last_clean":"2024-04-26T20:24:12.040181+0000","last_became_active":"2024-04-26T20:24:09.404472+0000","last_became_peered":"2024-04-26T20:24:09.404472+0000","last_unstale":"2024-04-26T20:24:12.040181+0000","last_undegraded":"2024-04-26T20:24:12.040181+0000","last_fullsized":"2024-04-26T20:24:12.040181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:33.926464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040051+0000","last_change":"2024-04-26T20:24:10.416286+0000","last_active":"2024-04-26T20:24:12.040051+0000","last_peered":"2024-04-26T20:24:12.040051+0000","last_clean":"2024-04-26T20:24:12.040051+0000","last_became_active":"2024-04-26T20:24:10.415996+0000","last_became_peered":"2024-04-26T20:24:10.415996+0000","last_unstale":"2024-04-26T20:24:12.040051+0000","last_undegraded":"2024-04-26T20:24:12.040051+0000","last_fullsized":"2024-04-26T20:24:12.040051+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.476611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412825+0000","last_change":"2024-04-26T20:24:09.407251+0000","last_active":"2024-04-26T20:24:11.412825+0000","last_peered":"2024-04-26T20:24:11.412825+0000","last_clean":"2024-04-26T20:24:11.412825+0000","last_became_active":"2024-04-26T20:24:09.406952+0000","last_became_peered":"2024-04-26T20:24:09.406952+0000","last_unstale":"2024-04-26T20:24:11.412825+0000","last_undegraded":"2024-04-26T20:24:11.412825+0000","last_fullsized":"2024-04-26T20:24:11.412825+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:27.816650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039904+0000","last_change":"2024-04-26T20:24:10.411680+0000","last_active":"2024-04-26T20:24:12.039904+0000","last_peered":"2024-04-26T20:24:12.039904+0000","last_clean":"2024-04-26T20:24:12.039904+0000","last_became_active":"2024-04-26T20:24:10.411311+0000","last_became_peered":"2024-04-26T20:24:10.411311+0000","last_unstale":"2024-04-26T20:24:12.039904+0000","last_undegraded":"2024-04-26T20:24:12.039904+0000","last_fullsized":"2024-04-26T20:24:12.039904+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:18.426587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413343+0000","last_change":"2024-04-26T20:24:09.400986+0000","last_active":"2024-04-26T20:24:11.413343+0000","last_peered":"2024-04-26T20:24:11.413343+0000","last_clean":"2024-04-26T20:24:11.413343+0000","last_became_active":"2024-04-26T20:24:09.400684+0000","last_became_peered":"2024-04-26T20:24:09.400684+0000","last_unstale":"2024-04-26T20:24:11.413343+0000","last_undegraded":"2024-04-26T20:24:11.413343+0000","last_fullsized":"2024-04-26T20:24:11.413343+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:29.072522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413351+0000","last_change":"2024-04-26T20:24:10.414112+0000","last_active":"2024-04-26T20:24:11.413351+0000","last_peered":"2024-04-26T20:24:11.413351+0000","last_clean":"2024-04-26T20:24:11.413351+0000","last_became_active":"2024-04-26T20:24:10.413818+0000","last_became_peered":"2024-04-26T20:24:10.413818+0000","last_unstale":"2024-04-26T20:24:11.413351+0000","last_undegraded":"2024-04-26T20:24:11.413351+0000","last_fullsized":"2024-04-26T20:24:11.413351+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:49.172379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413585+0000","last_change":"2024-04-26T20:24:09.402853+0000","last_active":"2024-04-26T20:24:11.413585+0000","last_peered":"2024-04-26T20:24:11.413585+0000","last_clean":"2024-04-26T20:24:11.413585+0000","last_became_active":"2024-04-26T20:24:09.402559+0000","last_became_peered":"2024-04-26T20:24:09.402559+0000","last_unstale":"2024-04-26T20:24:11.413585+0000","last_undegraded":"2024-04-26T20:24:11.413585+0000","last_fullsized":"2024-04-26T20:24:11.413585+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:06.566389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611286+0000","last_change":"2024-04-26T20:24:10.409357+0000","last_active":"2024-04-26T20:24:11.611286+0000","last_peered":"2024-04-26T20:24:11.611286+0000","last_clean":"2024-04-26T20:24:11.611286+0000","last_became_active":"2024-04-26T20:24:10.409018+0000","last_became_peered":"2024-04-26T20:24:10.409018+0000","last_unstale":"2024-04-26T20:24:11.611286+0000","last_undegraded":"2024-04-26T20:24:11.611286+0000","last_fullsized":"2024-04-26T20:24:11.611286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:28.821951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412627+0000","last_change":"2024-04-26T20:24:09.407446+0000","last_active":"2024-04-26T20:24:11.412627+0000","last_peered":"2024-04-26T20:24:11.412627+0000","last_clean":"2024-04-26T20:24:11.412627+0000","last_became_active":"2024-04-26T20:24:09.407138+0000","last_became_peered":"2024-04-26T20:24:09.407138+0000","last_unstale":"2024-04-26T20:24:11.412627+0000","last_undegraded":"2024-04-26T20:24:11.412627+0000","last_fullsized":"2024-04-26T20:24:11.412627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:05.750310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039473+0000","last_change":"2024-04-26T20:24:10.416903+0000","last_active":"2024-04-26T20:24:12.039473+0000","last_peered":"2024-04-26T20:24:12.039473+0000","last_clean":"2024-04-26T20:24:12.039473+0000","last_became_active":"2024-04-26T20:24:10.416080+0000","last_became_peered":"2024-04-26T20:24:10.416080+0000","last_unstale":"2024-04-26T20:24:12.039473+0000","last_undegraded":"2024-04-26T20:24:12.039473+0000","last_fullsized":"2024-04-26T20:24:12.039473+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:57.904889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.411829+0000","last_change":"2024-04-26T20:24:09.406182+0000","last_active":"2024-04-26T20:24:11.411829+0000","last_peered":"2024-04-26T20:24:11.411829+0000","last_clean":"2024-04-26T20:24:11.411829+0000","last_became_active":"2024-04-26T20:24:09.405719+0000","last_became_peered":"2024-04-26T20:24:09.405719+0000","last_unstale":"2024-04-26T20:24:11.411829+0000","last_undegraded":"2024-04-26T20:24:11.411829+0000","last_fullsized":"2024-04-26T20:24:11.411829+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:45.757564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.411806+0000","last_change":"2024-04-26T20:24:10.417503+0000","last_active":"2024-04-26T20:24:11.411806+0000","last_peered":"2024-04-26T20:24:11.411806+0000","last_clean":"2024-04-26T20:24:11.411806+0000","last_became_active":"2024-04-26T20:24:10.417215+0000","last_became_peered":"2024-04-26T20:24:10.417215+0000","last_unstale":"2024-04-26T20:24:11.411806+0000","last_undegraded":"2024-04-26T20:24:11.411806+0000","last_fullsized":"2024-04-26T20:24:11.411806+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:35.112499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.537962+0000","last_change":"2024-04-26T20:24:09.407652+0000","last_active":"2024-04-26T20:24:11.537962+0000","last_peered":"2024-04-26T20:24:11.537962+0000","last_clean":"2024-04-26T20:24:11.537962+0000","last_became_active":"2024-04-26T20:24:09.407323+0000","last_became_peered":"2024-04-26T20:24:09.407323+0000","last_unstale":"2024-04-26T20:24:11.537962+0000","last_undegraded":"2024-04-26T20:24:11.537962+0000","last_fullsized":"2024-04-26T20:24:11.537962+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:34.472483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040661+0000","last_change":"2024-04-26T20:24:10.418542+0000","last_active":"2024-04-26T20:24:12.040661+0000","last_peered":"2024-04-26T20:24:12.040661+0000","last_clean":"2024-04-26T20:24:12.040661+0000","last_became_active":"2024-04-26T20:24:10.418266+0000","last_became_peered":"2024-04-26T20:24:10.418266+0000","last_unstale":"2024-04-26T20:24:12.040661+0000","last_undegraded":"2024-04-26T20:24:12.040661+0000","last_fullsized":"2024-04-26T20:24:12.040661+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:21.585036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413271+0000","last_change":"2024-04-26T20:24:09.403967+0000","last_active":"2024-04-26T20:24:11.413271+0000","last_peered":"2024-04-26T20:24:11.413271+0000","last_clean":"2024-04-26T20:24:11.413271+0000","last_became_active":"2024-04-26T20:24:09.403658+0000","last_became_peered":"2024-04-26T20:24:09.403658+0000","last_unstale":"2024-04-26T20:24:11.413271+0000","last_undegraded":"2024-04-26T20:24:11.413271+0000","last_fullsized":"2024-04-26T20:24:11.413271+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:35.406719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412399+0000","last_change":"2024-04-26T20:24:10.407722+0000","last_active":"2024-04-26T20:24:11.412399+0000","last_peered":"2024-04-26T20:24:11.412399+0000","last_clean":"2024-04-26T20:24:11.412399+0000","last_became_active":"2024-04-26T20:24:10.407353+0000","last_became_peered":"2024-04-26T20:24:10.407353+0000","last_unstale":"2024-04-26T20:24:11.412399+0000","last_undegraded":"2024-04-26T20:24:11.412399+0000","last_fullsized":"2024-04-26T20:24:11.412399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:22.506318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538245+0000","last_change":"2024-04-26T20:24:09.408946+0000","last_active":"2024-04-26T20:24:11.538245+0000","last_peered":"2024-04-26T20:24:11.538245+0000","last_clean":"2024-04-26T20:24:11.538245+0000","last_became_active":"2024-04-26T20:24:09.408661+0000","last_became_peered":"2024-04-26T20:24:09.408661+0000","last_unstale":"2024-04-26T20:24:11.538245+0000","last_undegraded":"2024-04-26T20:24:11.538245+0000","last_fullsized":"2024-04-26T20:24:11.538245+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:29.072522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538176+0000","last_change":"2024-04-26T20:24:10.414873+0000","last_active":"2024-04-26T20:24:11.538176+0000","last_peered":"2024-04-26T20:24:11.538176+0000","last_clean":"2024-04-26T20:24:11.538176+0000","last_became_active":"2024-04-26T20:24:10.414602+0000","last_became_peered":"2024-04-26T20:24:10.414602+0000","last_unstale":"2024-04-26T20:24:11.538176+0000","last_undegraded":"2024-04-26T20:24:11.538176+0000","last_fullsized":"2024-04-26T20:24:11.538176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:53.099897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611177+0000","last_change":"2024-04-26T20:24:09.410720+0000","last_active":"2024-04-26T20:24:11.611177+0000","last_peered":"2024-04-26T20:24:11.611177+0000","last_clean":"2024-04-26T20:24:11.611177+0000","last_became_active":"2024-04-26T20:24:09.410423+0000","last_became_peered":"2024-04-26T20:24:09.410423+0000","last_unstale":"2024-04-26T20:24:11.611177+0000","last_undegraded":"2024-04-26T20:24:11.611177+0000","last_fullsized":"2024-04-26T20:24:11.611177+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:26.565983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413492+0000","last_change":"2024-04-26T20:24:10.415571+0000","last_active":"2024-04-26T20:24:11.413492+0000","last_peered":"2024-04-26T20:24:11.413492+0000","last_clean":"2024-04-26T20:24:11.413492+0000","last_became_active":"2024-04-26T20:24:10.415288+0000","last_became_peered":"2024-04-26T20:24:10.415288+0000","last_unstale":"2024-04-26T20:24:11.413492+0000","last_undegraded":"2024-04-26T20:24:11.413492+0000","last_fullsized":"2024-04-26T20:24:11.413492+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:16.157335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538171+0000","last_change":"2024-04-26T20:24:09.401361+0000","last_active":"2024-04-26T20:24:11.538171+0000","last_peered":"2024-04-26T20:24:11.538171+0000","last_clean":"2024-04-26T20:24:11.538171+0000","last_became_active":"2024-04-26T20:24:09.401086+0000","last_became_peered":"2024-04-26T20:24:09.401086+0000","last_unstale":"2024-04-26T20:24:11.538171+0000","last_undegraded":"2024-04-26T20:24:11.538171+0000","last_fullsized":"2024-04-26T20:24:11.538171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:50.422477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413350+0000","last_change":"2024-04-26T20:24:10.415802+0000","last_active":"2024-04-26T20:24:11.413350+0000","last_peered":"2024-04-26T20:24:11.413350+0000","last_clean":"2024-04-26T20:24:11.413350+0000","last_became_active":"2024-04-26T20:24:10.415467+0000","last_became_peered":"2024-04-26T20:24:10.415467+0000","last_unstale":"2024-04-26T20:24:11.413350+0000","last_undegraded":"2024-04-26T20:24:11.413350+0000","last_fullsized":"2024-04-26T20:24:11.413350+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:50.472802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039643+0000","last_change":"2024-04-26T20:24:09.402277+0000","last_active":"2024-04-26T20:24:12.039643+0000","last_peered":"2024-04-26T20:24:12.039643+0000","last_clean":"2024-04-26T20:24:12.039643+0000","last_became_active":"2024-04-26T20:24:09.402003+0000","last_became_peered":"2024-04-26T20:24:09.402003+0000","last_unstale":"2024-04-26T20:24:12.039643+0000","last_undegraded":"2024-04-26T20:24:12.039643+0000","last_fullsized":"2024-04-26T20:24:12.039643+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:46.184964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412749+0000","last_change":"2024-04-26T20:24:10.413990+0000","last_active":"2024-04-26T20:24:11.412749+0000","last_peered":"2024-04-26T20:24:11.412749+0000","last_clean":"2024-04-26T20:24:11.412749+0000","last_became_active":"2024-04-26T20:24:10.413726+0000","last_became_peered":"2024-04-26T20:24:10.413726+0000","last_unstale":"2024-04-26T20:24:11.412749+0000","last_undegraded":"2024-04-26T20:24:11.412749+0000","last_fullsized":"2024-04-26T20:24:11.412749+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:09.962717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611200+0000","last_change":"2024-04-26T20:24:09.401668+0000","last_active":"2024-04-26T20:24:11.611200+0000","last_peered":"2024-04-26T20:24:11.611200+0000","last_clean":"2024-04-26T20:24:11.611200+0000","last_became_active":"2024-04-26T20:24:09.401294+0000","last_became_peered":"2024-04-26T20:24:09.401294+0000","last_unstale":"2024-04-26T20:24:11.611200+0000","last_undegraded":"2024-04-26T20:24:11.611200+0000","last_fullsized":"2024-04-26T20:24:11.611200+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:20.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039605+0000","last_change":"2024-04-26T20:24:10.417123+0000","last_active":"2024-04-26T20:24:12.039605+0000","last_peered":"2024-04-26T20:24:12.039605+0000","last_clean":"2024-04-26T20:24:12.039605+0000","last_became_active":"2024-04-26T20:24:10.416828+0000","last_became_peered":"2024-04-26T20:24:10.416828+0000","last_unstale":"2024-04-26T20:24:12.039605+0000","last_undegraded":"2024-04-26T20:24:12.039605+0000","last_fullsized":"2024-04-26T20:24:12.039605+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:36.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412393+0000","last_change":"2024-04-26T20:24:09.412877+0000","last_active":"2024-04-26T20:24:11.412393+0000","last_peered":"2024-04-26T20:24:11.412393+0000","last_clean":"2024-04-26T20:24:11.412393+0000","last_became_active":"2024-04-26T20:24:09.412615+0000","last_became_peered":"2024-04-26T20:24:09.412615+0000","last_unstale":"2024-04-26T20:24:11.412393+0000","last_undegraded":"2024-04-26T20:24:11.412393+0000","last_fullsized":"2024-04-26T20:24:11.412393+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:02.144918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412604+0000","last_change":"2024-04-26T20:24:10.411821+0000","last_active":"2024-04-26T20:24:11.412604+0000","last_peered":"2024-04-26T20:24:11.412604+0000","last_clean":"2024-04-26T20:24:11.412604+0000","last_became_active":"2024-04-26T20:24:10.411507+0000","last_became_peered":"2024-04-26T20:24:10.411507+0000","last_unstale":"2024-04-26T20:24:11.412604+0000","last_undegraded":"2024-04-26T20:24:11.412604+0000","last_fullsized":"2024-04-26T20:24:11.412604+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:36.412020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413112+0000","last_change":"2024-04-26T20:24:09.407563+0000","last_active":"2024-04-26T20:24:11.413112+0000","last_peered":"2024-04-26T20:24:11.413112+0000","last_clean":"2024-04-26T20:24:11.413112+0000","last_became_active":"2024-04-26T20:24:09.407184+0000","last_became_peered":"2024-04-26T20:24:09.407184+0000","last_unstale":"2024-04-26T20:24:11.413112+0000","last_undegraded":"2024-04-26T20:24:11.413112+0000","last_fullsized":"2024-04-26T20:24:11.413112+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:43.407221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":219,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.537742+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:24:11.537742+0000","last_peered":"2024-04-26T20:24:11.537742+0000","last_clean":"2024-04-26T20:24:11.537742+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:24:11.537742+0000","last_undegraded":"2024-04-26T20:24:11.537742+0000","last_fullsized":"2024-04-26T20:24:11.537742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":806315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":216529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":222227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":540672,"data_stored":217469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":552960,"data_stored":222795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":217491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":222227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":812559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:24:13.424 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:24:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.560+0000 7f43455ac640 1 Processor -- start 2024-04-26T20:24:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.560+0000 7f43455ac640 1 -- start start 2024-04-26T20:24:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.560+0000 7f43455ac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4340162e40 0x7f43400d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.560+0000 7f43455ac640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f43400b2050 0x7f43400cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.560+0000 7f43455ac640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f43400cfd70 0x7f43400ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.560+0000 7f43455ac640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4340163220 con 0x7f43400cfd70 2024-04-26T20:24:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.560+0000 7f43455ac640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f43400d0eb0 con 0x7f43400b2050 2024-04-26T20:24:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.560+0000 7f43455ac640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f43400d01a0 con 0x7f4340162e40 2024-04-26T20:24:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.561+0000 7f433f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f43400cfd70 0x7f43400ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.561+0000 7f433f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f43400cfd70 0x7f43400ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43900/0 (socket says 172.21.15.37:43900) 2024-04-26T20:24:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.561+0000 7f433f7fe640 1 -- 172.21.15.37:0/1466269813 learned_addr learned my addr 172.21.15.37:0/1466269813 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.561+0000 7f433e7fc640 1 --2- 172.21.15.37:0/1466269813 >> v2:172.21.15.134:3300/0 conn(0x7f43400b2050 0x7f43400cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.561+0000 7f433effd640 1 --2- 172.21.15.37:0/1466269813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4340162e40 0x7f43400d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.561+0000 7f433f7fe640 1 -- 172.21.15.37:0/1466269813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4340162e40 msgr2=0x7f43400d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.561+0000 7f433f7fe640 1 --2- 172.21.15.37:0/1466269813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4340162e40 0x7f43400d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.561+0000 7f433f7fe640 1 -- 172.21.15.37:0/1466269813 >> v2:172.21.15.134:3300/0 conn(0x7f43400b2050 msgr2=0x7f43400cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.561+0000 7f433f7fe640 1 --2- 172.21.15.37:0/1466269813 >> v2:172.21.15.134:3300/0 conn(0x7f43400b2050 0x7f43400cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.561+0000 7f433f7fe640 1 -- 172.21.15.37:0/1466269813 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43400b1d70 con 0x7f43400cfd70 2024-04-26T20:24:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f433f7fe640 1 --2- 172.21.15.37:0/1466269813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43400cfd70 0x7f43400ce010 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4340073b00 tx=0x7f433802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02dd303a03d2f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f433dffb640 1 -- 172.21.15.37:0/1466269813 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338036070 con 0x7f43400cfd70 2024-04-26T20:24:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f433dffb640 1 -- 172.21.15.37:0/1466269813 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4338031600 con 0x7f43400cfd70 2024-04-26T20:24:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f433dffb640 1 -- 172.21.15.37:0/1466269813 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43380319d0 con 0x7f43400cfd70 2024-04-26T20:24:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f43455ac640 1 -- 172.21.15.37:0/1466269813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43400cfd70 msgr2=0x7f43400ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f43455ac640 1 --2- 172.21.15.37:0/1466269813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43400cfd70 0x7f43400ce010 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4340073b00 tx=0x7f433802a610 comp rx=0 tx=0).stop 2024-04-26T20:24:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f43455ac640 1 -- 172.21.15.37:0/1466269813 shutdown_connections 2024-04-26T20:24:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f43455ac640 1 --2- 172.21.15.37:0/1466269813 >> v2:172.21.15.134:3300/0 conn(0x7f43400b2050 0x7f43400cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f43455ac640 1 --2- 172.21.15.37:0/1466269813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43400cfd70 0x7f43400ce010 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f43455ac640 1 --2- 172.21.15.37:0/1466269813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4340162e40 0x7f43400d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f43455ac640 1 -- 172.21.15.37:0/1466269813 >> 172.21.15.37:0/1466269813 conn(0x7f434009ba50 msgr2=0x7f434009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f43455ac640 1 -- 172.21.15.37:0/1466269813 shutdown_connections 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.562+0000 7f43455ac640 1 -- 172.21.15.37:0/1466269813 wait complete. 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.564+0000 7f43455ac640 1 Processor -- start 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.564+0000 7f43455ac640 1 -- start start 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.564+0000 7f43455ac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43400b2050 0x7f4340151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.564+0000 7f43455ac640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43400cfd70 0x7f43400cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.564+0000 7f43455ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4340162e40 0x7f434015f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.564+0000 7f43455ac640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43400d1f20 con 0x7f4340162e40 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.564+0000 7f43455ac640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43400d1c40 con 0x7f43400cfd70 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.564+0000 7f43455ac640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f43400b24b0 con 0x7f43400b2050 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.564+0000 7f433f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4340162e40 0x7f434015f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f433f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4340162e40 0x7f434015f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43916/0 (socket says 172.21.15.37:43916) 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f433f7fe640 1 -- 172.21.15.37:0/1548573651 learned_addr learned my addr 172.21.15.37:0/1548573651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f433e7fc640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43400cfd70 0x7f43400cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f433effd640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43400b2050 0x7f4340151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f433f7fe640 1 -- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43400b2050 msgr2=0x7f4340151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f433f7fe640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43400b2050 0x7f4340151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f433f7fe640 1 -- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43400cfd70 msgr2=0x7f43400cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f433f7fe640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43400cfd70 0x7f43400cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f433f7fe640 1 -- 172.21.15.37:0/1548573651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4340151840 con 0x7f4340162e40 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f433f7fe640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4340162e40 0x7f434015f8e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4338009f40 tx=0x7f4338004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f43277fe640 1 -- 172.21.15.37:0/1548573651 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433803f020 con 0x7f4340162e40 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f43277fe640 1 -- 172.21.15.37:0/1548573651 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4338037040 con 0x7f4340162e40 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f43277fe640 1 -- 172.21.15.37:0/1548573651 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338036070 con 0x7f4340162e40 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f43455ac640 1 -- 172.21.15.37:0/1548573651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434015e840 con 0x7f4340162e40 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.565+0000 7f43455ac640 1 -- 172.21.15.37:0/1548573651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434015ee00 con 0x7f4340162e40 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.566+0000 7f43455ac640 1 -- 172.21.15.37:0/1548573651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f434006a360 con 0x7f4340162e40 2024-04-26T20:24:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.567+0000 7f43277fe640 1 -- 172.21.15.37:0/1548573651 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f4338042020 con 0x7f4340162e40 2024-04-26T20:24:13.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.567+0000 7f43277fe640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f432c06cbc0 0x7f432c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.567+0000 7f43277fe640 1 -- 172.21.15.37:0/1548573651 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f43380ac460 con 0x7f4340162e40 2024-04-26T20:24:13.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.567+0000 7f433effd640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f432c06cbc0 0x7f432c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.568+0000 7f433effd640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f432c06cbc0 0x7f432c06f080 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4330006fd0 tx=0x7f4330008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:13.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.571+0000 7f43277fe640 1 -- 172.21.15.37:0/1548573651 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4338049030 con 0x7f4340162e40 2024-04-26T20:24:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.752+0000 7f43455ac640 1 -- 172.21.15.37:0/1548573651 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f434015add0 con 0x7f432c06cbc0 2024-04-26T20:24:13.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.759+0000 7f43277fe640 1 -- 172.21.15.37:0/1548573651 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251956 (secure 0 0 0) 0x7f434015add0 con 0x7f432c06cbc0 2024-04-26T20:24:13.763 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:13.765 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:24:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 -- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f432c06cbc0 msgr2=0x7f432c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f432c06cbc0 0x7f432c06f080 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4330006fd0 tx=0x7f4330008040 comp rx=0 tx=0).stop 2024-04-26T20:24:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 -- 172.21.15.37:0/1548573651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4340162e40 msgr2=0x7f434015f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4340162e40 0x7f434015f8e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4338009f40 tx=0x7f4338004a40 comp rx=0 tx=0).stop 2024-04-26T20:24:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 -- 172.21.15.37:0/1548573651 shutdown_connections 2024-04-26T20:24:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4340162e40 0x7f434015f8e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43400cfd70 0x7f43400cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f432c06cbc0 0x7f432c06f080 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 --2- 172.21.15.37:0/1548573651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43400b2050 0x7f4340151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 -- 172.21.15.37:0/1548573651 >> 172.21.15.37:0/1548573651 conn(0x7f434009ba50 msgr2=0x7f434006eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 -- 172.21.15.37:0/1548573651 shutdown_connections 2024-04-26T20:24:13.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.762+0000 7f43455ac640 1 -- 172.21.15.37:0/1548573651 wait complete. 2024-04-26T20:24:13.775 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-04-26T20:24:13.368858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234068,"kb_used_data":5400,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754740652,"statfs":{"total":773094113280,"available":772854427648,"internally_reserved":0,"allocated":5529600,"data_stored":2937612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.985389"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538141+0000","last_change":"2024-04-26T20:24:10.412044+0000","last_active":"2024-04-26T20:24:11.538141+0000","last_peered":"2024-04-26T20:24:11.538141+0000","last_clean":"2024-04-26T20:24:11.538141+0000","last_became_active":"2024-04-26T20:24:10.411672+0000","last_became_peered":"2024-04-26T20:24:10.411672+0000","last_unstale":"2024-04-26T20:24:11.538141+0000","last_undegraded":"2024-04-26T20:24:11.538141+0000","last_fullsized":"2024-04-26T20:24:11.538141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:14.142583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413075+0000","last_change":"2024-04-26T20:24:10.420421+0000","last_active":"2024-04-26T20:24:11.413075+0000","last_peered":"2024-04-26T20:24:11.413075+0000","last_clean":"2024-04-26T20:24:11.413075+0000","last_became_active":"2024-04-26T20:24:10.419930+0000","last_became_peered":"2024-04-26T20:24:10.419930+0000","last_unstale":"2024-04-26T20:24:11.413075+0000","last_undegraded":"2024-04-26T20:24:11.413075+0000","last_fullsized":"2024-04-26T20:24:11.413075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:35.477784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039673+0000","last_change":"2024-04-26T20:24:10.420914+0000","last_active":"2024-04-26T20:24:12.039673+0000","last_peered":"2024-04-26T20:24:12.039673+0000","last_clean":"2024-04-26T20:24:12.039673+0000","last_became_active":"2024-04-26T20:24:10.420040+0000","last_became_peered":"2024-04-26T20:24:10.420040+0000","last_unstale":"2024-04-26T20:24:12.039673+0000","last_undegraded":"2024-04-26T20:24:12.039673+0000","last_fullsized":"2024-04-26T20:24:12.039673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:26:57.481629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412336+0000","last_change":"2024-04-26T20:24:10.410426+0000","last_active":"2024-04-26T20:24:11.412336+0000","last_peered":"2024-04-26T20:24:11.412336+0000","last_clean":"2024-04-26T20:24:11.412336+0000","last_became_active":"2024-04-26T20:24:10.410115+0000","last_became_peered":"2024-04-26T20:24:10.410115+0000","last_unstale":"2024-04-26T20:24:11.412336+0000","last_undegraded":"2024-04-26T20:24:11.412336+0000","last_fullsized":"2024-04-26T20:24:11.412336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:26:57.481629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412804+0000","last_change":"2024-04-26T20:24:10.411822+0000","last_active":"2024-04-26T20:24:11.412804+0000","last_peered":"2024-04-26T20:24:11.412804+0000","last_clean":"2024-04-26T20:24:11.412804+0000","last_became_active":"2024-04-26T20:24:10.411514+0000","last_became_peered":"2024-04-26T20:24:10.411514+0000","last_unstale":"2024-04-26T20:24:11.412804+0000","last_undegraded":"2024-04-26T20:24:11.412804+0000","last_fullsized":"2024-04-26T20:24:11.412804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:41.710250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413181+0000","last_change":"2024-04-26T20:24:10.414904+0000","last_active":"2024-04-26T20:24:11.413181+0000","last_peered":"2024-04-26T20:24:11.413181+0000","last_clean":"2024-04-26T20:24:11.413181+0000","last_became_active":"2024-04-26T20:24:10.414625+0000","last_became_peered":"2024-04-26T20:24:10.414625+0000","last_unstale":"2024-04-26T20:24:11.413181+0000","last_undegraded":"2024-04-26T20:24:11.413181+0000","last_fullsized":"2024-04-26T20:24:11.413181+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:14.142583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538012+0000","last_change":"2024-04-26T20:24:10.420989+0000","last_active":"2024-04-26T20:24:11.538012+0000","last_peered":"2024-04-26T20:24:11.538012+0000","last_clean":"2024-04-26T20:24:11.538012+0000","last_became_active":"2024-04-26T20:24:10.420621+0000","last_became_peered":"2024-04-26T20:24:10.420621+0000","last_unstale":"2024-04-26T20:24:11.538012+0000","last_undegraded":"2024-04-26T20:24:11.538012+0000","last_fullsized":"2024-04-26T20:24:11.538012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:48.412809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412007+0000","last_change":"2024-04-26T20:24:10.411191+0000","last_active":"2024-04-26T20:24:11.412007+0000","last_peered":"2024-04-26T20:24:11.412007+0000","last_clean":"2024-04-26T20:24:11.412007+0000","last_became_active":"2024-04-26T20:24:10.410886+0000","last_became_peered":"2024-04-26T20:24:10.410886+0000","last_unstale":"2024-04-26T20:24:11.412007+0000","last_undegraded":"2024-04-26T20:24:11.412007+0000","last_fullsized":"2024-04-26T20:24:11.412007+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:44.412522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412286+0000","last_change":"2024-04-26T20:24:10.418832+0000","last_active":"2024-04-26T20:24:11.412286+0000","last_peered":"2024-04-26T20:24:11.412286+0000","last_clean":"2024-04-26T20:24:11.412286+0000","last_became_active":"2024-04-26T20:24:10.418549+0000","last_became_peered":"2024-04-26T20:24:10.418549+0000","last_unstale":"2024-04-26T20:24:11.412286+0000","last_undegraded":"2024-04-26T20:24:11.412286+0000","last_fullsized":"2024-04-26T20:24:11.412286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:36.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412598+0000","last_change":"2024-04-26T20:24:10.416343+0000","last_active":"2024-04-26T20:24:11.412598+0000","last_peered":"2024-04-26T20:24:11.412598+0000","last_clean":"2024-04-26T20:24:11.412598+0000","last_became_active":"2024-04-26T20:24:10.415517+0000","last_became_peered":"2024-04-26T20:24:10.415517+0000","last_unstale":"2024-04-26T20:24:11.412598+0000","last_undegraded":"2024-04-26T20:24:11.412598+0000","last_fullsized":"2024-04-26T20:24:11.412598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:41.710250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.411991+0000","last_change":"2024-04-26T20:24:10.419554+0000","last_active":"2024-04-26T20:24:11.411991+0000","last_peered":"2024-04-26T20:24:11.411991+0000","last_clean":"2024-04-26T20:24:11.411991+0000","last_became_active":"2024-04-26T20:24:10.419263+0000","last_became_peered":"2024-04-26T20:24:10.419263+0000","last_unstale":"2024-04-26T20:24:11.411991+0000","last_undegraded":"2024-04-26T20:24:11.411991+0000","last_fullsized":"2024-04-26T20:24:11.411991+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:31.589088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412924+0000","last_change":"2024-04-26T20:24:10.416431+0000","last_active":"2024-04-26T20:24:11.412924+0000","last_peered":"2024-04-26T20:24:11.412924+0000","last_clean":"2024-04-26T20:24:11.412924+0000","last_became_active":"2024-04-26T20:24:10.416133+0000","last_became_peered":"2024-04-26T20:24:10.416133+0000","last_unstale":"2024-04-26T20:24:11.412924+0000","last_undegraded":"2024-04-26T20:24:11.412924+0000","last_fullsized":"2024-04-26T20:24:11.412924+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:36.412020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039573+0000","last_change":"2024-04-26T20:24:10.421697+0000","last_active":"2024-04-26T20:24:12.039573+0000","last_peered":"2024-04-26T20:24:12.039573+0000","last_clean":"2024-04-26T20:24:12.039573+0000","last_became_active":"2024-04-26T20:24:10.421262+0000","last_became_peered":"2024-04-26T20:24:10.421262+0000","last_unstale":"2024-04-26T20:24:12.039573+0000","last_undegraded":"2024-04-26T20:24:12.039573+0000","last_fullsized":"2024-04-26T20:24:12.039573+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:46.762865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412079+0000","last_change":"2024-04-26T20:24:10.418165+0000","last_active":"2024-04-26T20:24:11.412079+0000","last_peered":"2024-04-26T20:24:11.412079+0000","last_clean":"2024-04-26T20:24:11.412079+0000","last_became_active":"2024-04-26T20:24:10.417893+0000","last_became_peered":"2024-04-26T20:24:10.417893+0000","last_unstale":"2024-04-26T20:24:11.412079+0000","last_undegraded":"2024-04-26T20:24:11.412079+0000","last_fullsized":"2024-04-26T20:24:11.412079+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:06.755611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413083+0000","last_change":"2024-04-26T20:24:10.409013+0000","last_active":"2024-04-26T20:24:11.413083+0000","last_peered":"2024-04-26T20:24:11.413083+0000","last_clean":"2024-04-26T20:24:11.413083+0000","last_became_active":"2024-04-26T20:24:10.408383+0000","last_became_peered":"2024-04-26T20:24:10.408383+0000","last_unstale":"2024-04-26T20:24:11.413083+0000","last_undegraded":"2024-04-26T20:24:11.413083+0000","last_fullsized":"2024-04-26T20:24:11.413083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:29.253761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039694+0000","last_change":"2024-04-26T20:24:10.418862+0000","last_active":"2024-04-26T20:24:12.039694+0000","last_peered":"2024-04-26T20:24:12.039694+0000","last_clean":"2024-04-26T20:24:12.039694+0000","last_became_active":"2024-04-26T20:24:10.418589+0000","last_became_peered":"2024-04-26T20:24:10.418589+0000","last_unstale":"2024-04-26T20:24:12.039694+0000","last_undegraded":"2024-04-26T20:24:12.039694+0000","last_fullsized":"2024-04-26T20:24:12.039694+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:16.533192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040093+0000","last_change":"2024-04-26T20:24:10.415295+0000","last_active":"2024-04-26T20:24:12.040093+0000","last_peered":"2024-04-26T20:24:12.040093+0000","last_clean":"2024-04-26T20:24:12.040093+0000","last_became_active":"2024-04-26T20:24:10.415005+0000","last_became_peered":"2024-04-26T20:24:10.415005+0000","last_unstale":"2024-04-26T20:24:12.040093+0000","last_undegraded":"2024-04-26T20:24:12.040093+0000","last_fullsized":"2024-04-26T20:24:12.040093+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:59.031860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040298+0000","last_change":"2024-04-26T20:24:10.421863+0000","last_active":"2024-04-26T20:24:12.040298+0000","last_peered":"2024-04-26T20:24:12.040298+0000","last_clean":"2024-04-26T20:24:12.040298+0000","last_became_active":"2024-04-26T20:24:10.421570+0000","last_became_peered":"2024-04-26T20:24:10.421570+0000","last_unstale":"2024-04-26T20:24:12.040298+0000","last_undegraded":"2024-04-26T20:24:12.040298+0000","last_fullsized":"2024-04-26T20:24:12.040298+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:09.962717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040532+0000","last_change":"2024-04-26T20:24:10.425654+0000","last_active":"2024-04-26T20:24:12.040532+0000","last_peered":"2024-04-26T20:24:12.040532+0000","last_clean":"2024-04-26T20:24:12.040532+0000","last_became_active":"2024-04-26T20:24:10.425418+0000","last_became_peered":"2024-04-26T20:24:10.425418+0000","last_unstale":"2024-04-26T20:24:12.040532+0000","last_undegraded":"2024-04-26T20:24:12.040532+0000","last_fullsized":"2024-04-26T20:24:12.040532+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:47.190265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039790+0000","last_change":"2024-04-26T20:24:10.425637+0000","last_active":"2024-04-26T20:24:12.039790+0000","last_peered":"2024-04-26T20:24:12.039790+0000","last_clean":"2024-04-26T20:24:12.039790+0000","last_became_active":"2024-04-26T20:24:10.425294+0000","last_became_peered":"2024-04-26T20:24:10.425294+0000","last_unstale":"2024-04-26T20:24:12.039790+0000","last_undegraded":"2024-04-26T20:24:12.039790+0000","last_fullsized":"2024-04-26T20:24:12.039790+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:03.150219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413190+0000","last_change":"2024-04-26T20:24:10.416227+0000","last_active":"2024-04-26T20:24:11.413190+0000","last_peered":"2024-04-26T20:24:11.413190+0000","last_clean":"2024-04-26T20:24:11.413190+0000","last_became_active":"2024-04-26T20:24:10.415783+0000","last_became_peered":"2024-04-26T20:24:10.415783+0000","last_unstale":"2024-04-26T20:24:11.413190+0000","last_undegraded":"2024-04-26T20:24:11.413190+0000","last_fullsized":"2024-04-26T20:24:11.413190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:53.099897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412388+0000","last_change":"2024-04-26T20:24:10.412142+0000","last_active":"2024-04-26T20:24:11.412388+0000","last_peered":"2024-04-26T20:24:11.412388+0000","last_clean":"2024-04-26T20:24:11.412388+0000","last_became_active":"2024-04-26T20:24:10.411703+0000","last_became_peered":"2024-04-26T20:24:10.411703+0000","last_unstale":"2024-04-26T20:24:11.412388+0000","last_undegraded":"2024-04-26T20:24:11.412388+0000","last_fullsized":"2024-04-26T20:24:11.412388+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:21.440553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040288+0000","last_change":"2024-04-26T20:24:10.412984+0000","last_active":"2024-04-26T20:24:12.040288+0000","last_peered":"2024-04-26T20:24:12.040288+0000","last_clean":"2024-04-26T20:24:12.040288+0000","last_became_active":"2024-04-26T20:24:10.412668+0000","last_became_peered":"2024-04-26T20:24:10.412668+0000","last_unstale":"2024-04-26T20:24:12.040288+0000","last_undegraded":"2024-04-26T20:24:12.040288+0000","last_fullsized":"2024-04-26T20:24:12.040288+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:30.101850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039884+0000","last_change":"2024-04-26T20:24:10.413134+0000","last_active":"2024-04-26T20:24:12.039884+0000","last_peered":"2024-04-26T20:24:12.039884+0000","last_clean":"2024-04-26T20:24:12.039884+0000","last_became_active":"2024-04-26T20:24:10.412804+0000","last_became_peered":"2024-04-26T20:24:10.412804+0000","last_unstale":"2024-04-26T20:24:12.039884+0000","last_undegraded":"2024-04-26T20:24:12.039884+0000","last_fullsized":"2024-04-26T20:24:12.039884+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:03.150219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040559+0000","last_change":"2024-04-26T20:24:10.411413+0000","last_active":"2024-04-26T20:24:12.040559+0000","last_peered":"2024-04-26T20:24:12.040559+0000","last_clean":"2024-04-26T20:24:12.040559+0000","last_became_active":"2024-04-26T20:24:10.411160+0000","last_became_peered":"2024-04-26T20:24:10.411160+0000","last_unstale":"2024-04-26T20:24:12.040559+0000","last_undegraded":"2024-04-26T20:24:12.040559+0000","last_fullsized":"2024-04-26T20:24:12.040559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:27.571285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039546+0000","last_change":"2024-04-26T20:24:10.420681+0000","last_active":"2024-04-26T20:24:12.039546+0000","last_peered":"2024-04-26T20:24:12.039546+0000","last_clean":"2024-04-26T20:24:12.039546+0000","last_became_active":"2024-04-26T20:24:10.420282+0000","last_became_peered":"2024-04-26T20:24:10.420282+0000","last_unstale":"2024-04-26T20:24:12.039546+0000","last_undegraded":"2024-04-26T20:24:12.039546+0000","last_fullsized":"2024-04-26T20:24:12.039546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:28.586657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039863+0000","last_change":"2024-04-26T20:24:10.411128+0000","last_active":"2024-04-26T20:24:12.039863+0000","last_peered":"2024-04-26T20:24:12.039863+0000","last_clean":"2024-04-26T20:24:12.039863+0000","last_became_active":"2024-04-26T20:24:10.410862+0000","last_became_peered":"2024-04-26T20:24:10.410862+0000","last_unstale":"2024-04-26T20:24:12.039863+0000","last_undegraded":"2024-04-26T20:24:12.039863+0000","last_fullsized":"2024-04-26T20:24:12.039863+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:21.440553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412990+0000","last_change":"2024-04-26T20:24:10.415403+0000","last_active":"2024-04-26T20:24:11.412990+0000","last_peered":"2024-04-26T20:24:11.412990+0000","last_clean":"2024-04-26T20:24:11.412990+0000","last_became_active":"2024-04-26T20:24:10.415147+0000","last_became_peered":"2024-04-26T20:24:10.415147+0000","last_unstale":"2024-04-26T20:24:11.412990+0000","last_undegraded":"2024-04-26T20:24:11.412990+0000","last_fullsized":"2024-04-26T20:24:11.412990+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:17.113224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040356+0000","last_change":"2024-04-26T20:24:10.410620+0000","last_active":"2024-04-26T20:24:12.040356+0000","last_peered":"2024-04-26T20:24:12.040356+0000","last_clean":"2024-04-26T20:24:12.040356+0000","last_became_active":"2024-04-26T20:24:10.410259+0000","last_became_peered":"2024-04-26T20:24:10.410259+0000","last_unstale":"2024-04-26T20:24:12.040356+0000","last_undegraded":"2024-04-26T20:24:12.040356+0000","last_fullsized":"2024-04-26T20:24:12.040356+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:35.010247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039526+0000","last_change":"2024-04-26T20:24:10.413050+0000","last_active":"2024-04-26T20:24:12.039526+0000","last_peered":"2024-04-26T20:24:12.039526+0000","last_clean":"2024-04-26T20:24:12.039526+0000","last_became_active":"2024-04-26T20:24:10.412454+0000","last_became_peered":"2024-04-26T20:24:10.412454+0000","last_unstale":"2024-04-26T20:24:12.039526+0000","last_undegraded":"2024-04-26T20:24:12.039526+0000","last_fullsized":"2024-04-26T20:24:12.039526+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:43.189695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413195+0000","last_change":"2024-04-26T20:24:10.409258+0000","last_active":"2024-04-26T20:24:11.413195+0000","last_peered":"2024-04-26T20:24:11.413195+0000","last_clean":"2024-04-26T20:24:11.413195+0000","last_became_active":"2024-04-26T20:24:10.408741+0000","last_became_peered":"2024-04-26T20:24:10.408741+0000","last_unstale":"2024-04-26T20:24:11.413195+0000","last_undegraded":"2024-04-26T20:24:11.413195+0000","last_fullsized":"2024-04-26T20:24:11.413195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:51.460143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039414+0000","last_change":"2024-04-26T20:24:10.416638+0000","last_active":"2024-04-26T20:24:12.039414+0000","last_peered":"2024-04-26T20:24:12.039414+0000","last_clean":"2024-04-26T20:24:12.039414+0000","last_became_active":"2024-04-26T20:24:10.415734+0000","last_became_peered":"2024-04-26T20:24:10.415734+0000","last_unstale":"2024-04-26T20:24:12.039414+0000","last_undegraded":"2024-04-26T20:24:12.039414+0000","last_fullsized":"2024-04-26T20:24:12.039414+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:10.932936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611360+0000","last_change":"2024-04-26T20:24:10.416676+0000","last_active":"2024-04-26T20:24:11.611360+0000","last_peered":"2024-04-26T20:24:11.611360+0000","last_clean":"2024-04-26T20:24:11.611360+0000","last_became_active":"2024-04-26T20:24:10.416413+0000","last_became_peered":"2024-04-26T20:24:10.416413+0000","last_unstale":"2024-04-26T20:24:11.611360+0000","last_undegraded":"2024-04-26T20:24:11.611360+0000","last_fullsized":"2024-04-26T20:24:11.611360+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:31.589088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413644+0000","last_change":"2024-04-26T20:24:09.400089+0000","last_active":"2024-04-26T20:24:11.413644+0000","last_peered":"2024-04-26T20:24:11.413644+0000","last_clean":"2024-04-26T20:24:11.413644+0000","last_became_active":"2024-04-26T20:24:09.399799+0000","last_became_peered":"2024-04-26T20:24:09.399799+0000","last_unstale":"2024-04-26T20:24:11.413644+0000","last_undegraded":"2024-04-26T20:24:11.413644+0000","last_fullsized":"2024-04-26T20:24:11.413644+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:34.472483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040692+0000","last_change":"2024-04-26T20:24:10.407406+0000","last_active":"2024-04-26T20:24:12.040692+0000","last_peered":"2024-04-26T20:24:12.040692+0000","last_clean":"2024-04-26T20:24:12.040692+0000","last_became_active":"2024-04-26T20:24:10.407106+0000","last_became_peered":"2024-04-26T20:24:10.407106+0000","last_unstale":"2024-04-26T20:24:12.040692+0000","last_undegraded":"2024-04-26T20:24:12.040692+0000","last_fullsized":"2024-04-26T20:24:12.040692+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:34.931765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040070+0000","last_change":"2024-04-26T20:24:09.402566+0000","last_active":"2024-04-26T20:24:12.040070+0000","last_peered":"2024-04-26T20:24:12.040070+0000","last_clean":"2024-04-26T20:24:12.040070+0000","last_became_active":"2024-04-26T20:24:09.402281+0000","last_became_peered":"2024-04-26T20:24:09.402281+0000","last_unstale":"2024-04-26T20:24:12.040070+0000","last_undegraded":"2024-04-26T20:24:12.040070+0000","last_fullsized":"2024-04-26T20:24:12.040070+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:20.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413182+0000","last_change":"2024-04-26T20:24:10.418192+0000","last_active":"2024-04-26T20:24:11.413182+0000","last_peered":"2024-04-26T20:24:11.413182+0000","last_clean":"2024-04-26T20:24:11.413182+0000","last_became_active":"2024-04-26T20:24:10.417863+0000","last_became_peered":"2024-04-26T20:24:10.417863+0000","last_unstale":"2024-04-26T20:24:11.413182+0000","last_undegraded":"2024-04-26T20:24:11.413182+0000","last_fullsized":"2024-04-26T20:24:11.413182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:45.455882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611441+0000","last_change":"2024-04-26T20:24:09.410058+0000","last_active":"2024-04-26T20:24:11.611441+0000","last_peered":"2024-04-26T20:24:11.611441+0000","last_clean":"2024-04-26T20:24:11.611441+0000","last_became_active":"2024-04-26T20:24:09.409706+0000","last_became_peered":"2024-04-26T20:24:09.409706+0000","last_unstale":"2024-04-26T20:24:11.611441+0000","last_undegraded":"2024-04-26T20:24:11.611441+0000","last_fullsized":"2024-04-26T20:24:11.611441+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:33.926464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611493+0000","last_change":"2024-04-26T20:24:10.410080+0000","last_active":"2024-04-26T20:24:11.611493+0000","last_peered":"2024-04-26T20:24:11.611493+0000","last_clean":"2024-04-26T20:24:11.611493+0000","last_became_active":"2024-04-26T20:24:10.409736+0000","last_became_peered":"2024-04-26T20:24:10.409736+0000","last_unstale":"2024-04-26T20:24:11.611493+0000","last_undegraded":"2024-04-26T20:24:11.611493+0000","last_fullsized":"2024-04-26T20:24:11.611493+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:44.412522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.537922+0000","last_change":"2024-04-26T20:24:09.405279+0000","last_active":"2024-04-26T20:24:11.537922+0000","last_peered":"2024-04-26T20:24:11.537922+0000","last_clean":"2024-04-26T20:24:11.537922+0000","last_became_active":"2024-04-26T20:24:09.404980+0000","last_became_peered":"2024-04-26T20:24:09.404980+0000","last_unstale":"2024-04-26T20:24:11.537922+0000","last_undegraded":"2024-04-26T20:24:11.537922+0000","last_fullsized":"2024-04-26T20:24:11.537922+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:44.450580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039493+0000","last_change":"2024-04-26T20:24:10.409268+0000","last_active":"2024-04-26T20:24:12.039493+0000","last_peered":"2024-04-26T20:24:12.039493+0000","last_clean":"2024-04-26T20:24:12.039493+0000","last_became_active":"2024-04-26T20:24:10.408997+0000","last_became_peered":"2024-04-26T20:24:10.408997+0000","last_unstale":"2024-04-26T20:24:12.039493+0000","last_undegraded":"2024-04-26T20:24:12.039493+0000","last_fullsized":"2024-04-26T20:24:12.039493+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:06.755611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413485+0000","last_change":"2024-04-26T20:24:09.405281+0000","last_active":"2024-04-26T20:24:11.413485+0000","last_peered":"2024-04-26T20:24:11.413485+0000","last_clean":"2024-04-26T20:24:11.413485+0000","last_became_active":"2024-04-26T20:24:09.404954+0000","last_became_peered":"2024-04-26T20:24:09.404954+0000","last_unstale":"2024-04-26T20:24:11.413485+0000","last_undegraded":"2024-04-26T20:24:11.413485+0000","last_fullsized":"2024-04-26T20:24:11.413485+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:50.422477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412148+0000","last_change":"2024-04-26T20:24:10.425832+0000","last_active":"2024-04-26T20:24:11.412148+0000","last_peered":"2024-04-26T20:24:11.412148+0000","last_clean":"2024-04-26T20:24:11.412148+0000","last_became_active":"2024-04-26T20:24:10.425361+0000","last_became_peered":"2024-04-26T20:24:10.425361+0000","last_unstale":"2024-04-26T20:24:11.412148+0000","last_undegraded":"2024-04-26T20:24:11.412148+0000","last_fullsized":"2024-04-26T20:24:11.412148+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:43.189695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040300+0000","last_change":"2024-04-26T20:24:09.408539+0000","last_active":"2024-04-26T20:24:12.040300+0000","last_peered":"2024-04-26T20:24:12.040300+0000","last_clean":"2024-04-26T20:24:12.040300+0000","last_became_active":"2024-04-26T20:24:09.408022+0000","last_became_peered":"2024-04-26T20:24:09.408022+0000","last_unstale":"2024-04-26T20:24:12.040300+0000","last_undegraded":"2024-04-26T20:24:12.040300+0000","last_fullsized":"2024-04-26T20:24:12.040300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:34.004945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.411807+0000","last_change":"2024-04-26T20:24:10.419051+0000","last_active":"2024-04-26T20:24:11.411807+0000","last_peered":"2024-04-26T20:24:11.411807+0000","last_clean":"2024-04-26T20:24:11.411807+0000","last_became_active":"2024-04-26T20:24:10.412887+0000","last_became_peered":"2024-04-26T20:24:10.412887+0000","last_unstale":"2024-04-26T20:24:11.411807+0000","last_undegraded":"2024-04-26T20:24:11.411807+0000","last_fullsized":"2024-04-26T20:24:11.411807+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:28.821951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412653+0000","last_change":"2024-04-26T20:24:09.407265+0000","last_active":"2024-04-26T20:24:11.412653+0000","last_peered":"2024-04-26T20:24:11.412653+0000","last_clean":"2024-04-26T20:24:11.412653+0000","last_became_active":"2024-04-26T20:24:09.406440+0000","last_became_peered":"2024-04-26T20:24:09.406440+0000","last_unstale":"2024-04-26T20:24:11.412653+0000","last_undegraded":"2024-04-26T20:24:11.412653+0000","last_fullsized":"2024-04-26T20:24:11.412653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:30.583786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040142+0000","last_change":"2024-04-26T20:24:10.412988+0000","last_active":"2024-04-26T20:24:12.040142+0000","last_peered":"2024-04-26T20:24:12.040142+0000","last_clean":"2024-04-26T20:24:12.040142+0000","last_became_active":"2024-04-26T20:24:10.412605+0000","last_became_peered":"2024-04-26T20:24:10.412605+0000","last_unstale":"2024-04-26T20:24:12.040142+0000","last_undegraded":"2024-04-26T20:24:12.040142+0000","last_fullsized":"2024-04-26T20:24:12.040142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:54.004667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412365+0000","last_change":"2024-04-26T20:24:09.404738+0000","last_active":"2024-04-26T20:24:11.412365+0000","last_peered":"2024-04-26T20:24:11.412365+0000","last_clean":"2024-04-26T20:24:11.412365+0000","last_became_active":"2024-04-26T20:24:09.404407+0000","last_became_peered":"2024-04-26T20:24:09.404407+0000","last_unstale":"2024-04-26T20:24:11.412365+0000","last_undegraded":"2024-04-26T20:24:11.412365+0000","last_fullsized":"2024-04-26T20:24:11.412365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:27.581356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039877+0000","last_change":"2024-04-26T20:24:10.421571+0000","last_active":"2024-04-26T20:24:12.039877+0000","last_peered":"2024-04-26T20:24:12.039877+0000","last_clean":"2024-04-26T20:24:12.039877+0000","last_became_active":"2024-04-26T20:24:10.421406+0000","last_became_peered":"2024-04-26T20:24:10.421406+0000","last_unstale":"2024-04-26T20:24:12.039877+0000","last_undegraded":"2024-04-26T20:24:12.039877+0000","last_fullsized":"2024-04-26T20:24:12.039877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:22.506318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040280+0000","last_change":"2024-04-26T20:24:09.408668+0000","last_active":"2024-04-26T20:24:12.040280+0000","last_peered":"2024-04-26T20:24:12.040280+0000","last_clean":"2024-04-26T20:24:12.040280+0000","last_became_active":"2024-04-26T20:24:09.408423+0000","last_became_peered":"2024-04-26T20:24:09.408423+0000","last_unstale":"2024-04-26T20:24:12.040280+0000","last_undegraded":"2024-04-26T20:24:12.040280+0000","last_fullsized":"2024-04-26T20:24:12.040280+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:26.565983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538065+0000","last_change":"2024-04-26T20:24:10.419074+0000","last_active":"2024-04-26T20:24:11.538065+0000","last_peered":"2024-04-26T20:24:11.538065+0000","last_clean":"2024-04-26T20:24:11.538065+0000","last_became_active":"2024-04-26T20:24:10.418701+0000","last_became_peered":"2024-04-26T20:24:10.418701+0000","last_unstale":"2024-04-26T20:24:11.538065+0000","last_undegraded":"2024-04-26T20:24:11.538065+0000","last_fullsized":"2024-04-26T20:24:11.538065+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:29.253761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412680+0000","last_change":"2024-04-26T20:24:09.409147+0000","last_active":"2024-04-26T20:24:11.412680+0000","last_peered":"2024-04-26T20:24:11.412680+0000","last_clean":"2024-04-26T20:24:11.412680+0000","last_became_active":"2024-04-26T20:24:09.408634+0000","last_became_peered":"2024-04-26T20:24:09.408634+0000","last_unstale":"2024-04-26T20:24:11.412680+0000","last_undegraded":"2024-04-26T20:24:11.412680+0000","last_fullsized":"2024-04-26T20:24:11.412680+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:17.421286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413428+0000","last_change":"2024-04-26T20:24:10.414946+0000","last_active":"2024-04-26T20:24:11.413428+0000","last_peered":"2024-04-26T20:24:11.413428+0000","last_clean":"2024-04-26T20:24:11.413428+0000","last_became_active":"2024-04-26T20:24:10.414630+0000","last_became_peered":"2024-04-26T20:24:10.414630+0000","last_unstale":"2024-04-26T20:24:11.413428+0000","last_undegraded":"2024-04-26T20:24:11.413428+0000","last_fullsized":"2024-04-26T20:24:11.413428+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:25.267112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538154+0000","last_change":"2024-04-26T20:24:09.408914+0000","last_active":"2024-04-26T20:24:11.538154+0000","last_peered":"2024-04-26T20:24:11.538154+0000","last_clean":"2024-04-26T20:24:11.538154+0000","last_became_active":"2024-04-26T20:24:09.408610+0000","last_became_peered":"2024-04-26T20:24:09.408610+0000","last_unstale":"2024-04-26T20:24:11.538154+0000","last_undegraded":"2024-04-26T20:24:11.538154+0000","last_fullsized":"2024-04-26T20:24:11.538154+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:06.566389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538022+0000","last_change":"2024-04-26T20:24:10.420829+0000","last_active":"2024-04-26T20:24:11.538022+0000","last_peered":"2024-04-26T20:24:11.538022+0000","last_clean":"2024-04-26T20:24:11.538022+0000","last_became_active":"2024-04-26T20:24:10.420568+0000","last_became_peered":"2024-04-26T20:24:10.420568+0000","last_unstale":"2024-04-26T20:24:11.538022+0000","last_undegraded":"2024-04-26T20:24:11.538022+0000","last_fullsized":"2024-04-26T20:24:11.538022+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:33.660671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039773+0000","last_change":"2024-04-26T20:24:09.409891+0000","last_active":"2024-04-26T20:24:12.039773+0000","last_peered":"2024-04-26T20:24:12.039773+0000","last_clean":"2024-04-26T20:24:12.039773+0000","last_became_active":"2024-04-26T20:24:09.409604+0000","last_became_peered":"2024-04-26T20:24:09.409604+0000","last_unstale":"2024-04-26T20:24:12.039773+0000","last_undegraded":"2024-04-26T20:24:12.039773+0000","last_fullsized":"2024-04-26T20:24:12.039773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:17.677447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040028+0000","last_change":"2024-04-26T20:24:10.425603+0000","last_active":"2024-04-26T20:24:12.040028+0000","last_peered":"2024-04-26T20:24:12.040028+0000","last_clean":"2024-04-26T20:24:12.040028+0000","last_became_active":"2024-04-26T20:24:10.425317+0000","last_became_peered":"2024-04-26T20:24:10.425317+0000","last_unstale":"2024-04-26T20:24:12.040028+0000","last_undegraded":"2024-04-26T20:24:12.040028+0000","last_fullsized":"2024-04-26T20:24:12.040028+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:43.697260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413479+0000","last_change":"2024-04-26T20:24:09.408958+0000","last_active":"2024-04-26T20:24:11.413479+0000","last_peered":"2024-04-26T20:24:11.413479+0000","last_clean":"2024-04-26T20:24:11.413479+0000","last_became_active":"2024-04-26T20:24:09.408684+0000","last_became_peered":"2024-04-26T20:24:09.408684+0000","last_unstale":"2024-04-26T20:24:11.413479+0000","last_undegraded":"2024-04-26T20:24:11.413479+0000","last_fullsized":"2024-04-26T20:24:11.413479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:44.450580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412523+0000","last_change":"2024-04-26T20:24:10.412257+0000","last_active":"2024-04-26T20:24:11.412523+0000","last_peered":"2024-04-26T20:24:11.412523+0000","last_clean":"2024-04-26T20:24:11.412523+0000","last_became_active":"2024-04-26T20:24:10.411429+0000","last_became_peered":"2024-04-26T20:24:10.411429+0000","last_unstale":"2024-04-26T20:24:11.412523+0000","last_undegraded":"2024-04-26T20:24:11.412523+0000","last_fullsized":"2024-04-26T20:24:11.412523+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:59.031860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538194+0000","last_change":"2024-04-26T20:24:09.406698+0000","last_active":"2024-04-26T20:24:11.538194+0000","last_peered":"2024-04-26T20:24:11.538194+0000","last_clean":"2024-04-26T20:24:11.538194+0000","last_became_active":"2024-04-26T20:24:09.406430+0000","last_became_peered":"2024-04-26T20:24:09.406430+0000","last_unstale":"2024-04-26T20:24:11.538194+0000","last_undegraded":"2024-04-26T20:24:11.538194+0000","last_fullsized":"2024-04-26T20:24:11.538194+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:16.107922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413041+0000","last_change":"2024-04-26T20:24:10.409367+0000","last_active":"2024-04-26T20:24:11.413041+0000","last_peered":"2024-04-26T20:24:11.413041+0000","last_clean":"2024-04-26T20:24:11.413041+0000","last_became_active":"2024-04-26T20:24:10.409060+0000","last_became_peered":"2024-04-26T20:24:10.409060+0000","last_unstale":"2024-04-26T20:24:11.413041+0000","last_undegraded":"2024-04-26T20:24:11.413041+0000","last_fullsized":"2024-04-26T20:24:11.413041+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:33.660671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413322+0000","last_change":"2024-04-26T20:24:09.408235+0000","last_active":"2024-04-26T20:24:11.413322+0000","last_peered":"2024-04-26T20:24:11.413322+0000","last_clean":"2024-04-26T20:24:11.413322+0000","last_became_active":"2024-04-26T20:24:09.407953+0000","last_became_peered":"2024-04-26T20:24:09.407953+0000","last_unstale":"2024-04-26T20:24:11.413322+0000","last_undegraded":"2024-04-26T20:24:11.413322+0000","last_fullsized":"2024-04-26T20:24:11.413322+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:16.107922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538183+0000","last_change":"2024-04-26T20:24:10.417833+0000","last_active":"2024-04-26T20:24:11.538183+0000","last_peered":"2024-04-26T20:24:11.538183+0000","last_clean":"2024-04-26T20:24:11.538183+0000","last_became_active":"2024-04-26T20:24:10.417473+0000","last_became_peered":"2024-04-26T20:24:10.417473+0000","last_unstale":"2024-04-26T20:24:11.538183+0000","last_undegraded":"2024-04-26T20:24:11.538183+0000","last_fullsized":"2024-04-26T20:24:11.538183+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:51.460143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":22,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412298+0000","last_change":"2024-04-26T20:24:09.410011+0000","last_active":"2024-04-26T20:24:11.412298+0000","last_peered":"2024-04-26T20:24:11.412298+0000","last_clean":"2024-04-26T20:24:11.412298+0000","last_became_active":"2024-04-26T20:24:09.409716+0000","last_became_peered":"2024-04-26T20:24:09.409716+0000","last_unstale":"2024-04-26T20:24:11.412298+0000","last_undegraded":"2024-04-26T20:24:11.412298+0000","last_fullsized":"2024-04-26T20:24:11.412298+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:15.527890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039598+0000","last_change":"2024-04-26T20:24:10.410349+0000","last_active":"2024-04-26T20:24:12.039598+0000","last_peered":"2024-04-26T20:24:12.039598+0000","last_clean":"2024-04-26T20:24:12.039598+0000","last_became_active":"2024-04-26T20:24:10.410033+0000","last_became_peered":"2024-04-26T20:24:10.410033+0000","last_unstale":"2024-04-26T20:24:12.039598+0000","last_undegraded":"2024-04-26T20:24:12.039598+0000","last_fullsized":"2024-04-26T20:24:12.039598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:16.533192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039623+0000","last_change":"2024-04-26T20:24:09.407042+0000","last_active":"2024-04-26T20:24:12.039623+0000","last_peered":"2024-04-26T20:24:12.039623+0000","last_clean":"2024-04-26T20:24:12.039623+0000","last_became_active":"2024-04-26T20:24:09.406397+0000","last_became_peered":"2024-04-26T20:24:09.406397+0000","last_unstale":"2024-04-26T20:24:12.039623+0000","last_undegraded":"2024-04-26T20:24:12.039623+0000","last_fullsized":"2024-04-26T20:24:12.039623+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:32.120531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412572+0000","last_change":"2024-04-26T20:24:10.422603+0000","last_active":"2024-04-26T20:24:11.412572+0000","last_peered":"2024-04-26T20:24:11.412572+0000","last_clean":"2024-04-26T20:24:11.412572+0000","last_became_active":"2024-04-26T20:24:10.422331+0000","last_became_peered":"2024-04-26T20:24:10.422331+0000","last_unstale":"2024-04-26T20:24:11.412572+0000","last_undegraded":"2024-04-26T20:24:11.412572+0000","last_fullsized":"2024-04-26T20:24:11.412572+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:09.962717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040181+0000","last_change":"2024-04-26T20:24:09.404863+0000","last_active":"2024-04-26T20:24:12.040181+0000","last_peered":"2024-04-26T20:24:12.040181+0000","last_clean":"2024-04-26T20:24:12.040181+0000","last_became_active":"2024-04-26T20:24:09.404472+0000","last_became_peered":"2024-04-26T20:24:09.404472+0000","last_unstale":"2024-04-26T20:24:12.040181+0000","last_undegraded":"2024-04-26T20:24:12.040181+0000","last_fullsized":"2024-04-26T20:24:12.040181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:33.926464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040051+0000","last_change":"2024-04-26T20:24:10.416286+0000","last_active":"2024-04-26T20:24:12.040051+0000","last_peered":"2024-04-26T20:24:12.040051+0000","last_clean":"2024-04-26T20:24:12.040051+0000","last_became_active":"2024-04-26T20:24:10.415996+0000","last_became_peered":"2024-04-26T20:24:10.415996+0000","last_unstale":"2024-04-26T20:24:12.040051+0000","last_undegraded":"2024-04-26T20:24:12.040051+0000","last_fullsized":"2024-04-26T20:24:12.040051+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.476611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412825+0000","last_change":"2024-04-26T20:24:09.407251+0000","last_active":"2024-04-26T20:24:11.412825+0000","last_peered":"2024-04-26T20:24:11.412825+0000","last_clean":"2024-04-26T20:24:11.412825+0000","last_became_active":"2024-04-26T20:24:09.406952+0000","last_became_peered":"2024-04-26T20:24:09.406952+0000","last_unstale":"2024-04-26T20:24:11.412825+0000","last_undegraded":"2024-04-26T20:24:11.412825+0000","last_fullsized":"2024-04-26T20:24:11.412825+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:27.816650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039904+0000","last_change":"2024-04-26T20:24:10.411680+0000","last_active":"2024-04-26T20:24:12.039904+0000","last_peered":"2024-04-26T20:24:12.039904+0000","last_clean":"2024-04-26T20:24:12.039904+0000","last_became_active":"2024-04-26T20:24:10.411311+0000","last_became_peered":"2024-04-26T20:24:10.411311+0000","last_unstale":"2024-04-26T20:24:12.039904+0000","last_undegraded":"2024-04-26T20:24:12.039904+0000","last_fullsized":"2024-04-26T20:24:12.039904+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:18.426587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413343+0000","last_change":"2024-04-26T20:24:09.400986+0000","last_active":"2024-04-26T20:24:11.413343+0000","last_peered":"2024-04-26T20:24:11.413343+0000","last_clean":"2024-04-26T20:24:11.413343+0000","last_became_active":"2024-04-26T20:24:09.400684+0000","last_became_peered":"2024-04-26T20:24:09.400684+0000","last_unstale":"2024-04-26T20:24:11.413343+0000","last_undegraded":"2024-04-26T20:24:11.413343+0000","last_fullsized":"2024-04-26T20:24:11.413343+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:29.072522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413351+0000","last_change":"2024-04-26T20:24:10.414112+0000","last_active":"2024-04-26T20:24:11.413351+0000","last_peered":"2024-04-26T20:24:11.413351+0000","last_clean":"2024-04-26T20:24:11.413351+0000","last_became_active":"2024-04-26T20:24:10.413818+0000","last_became_peered":"2024-04-26T20:24:10.413818+0000","last_unstale":"2024-04-26T20:24:11.413351+0000","last_undegraded":"2024-04-26T20:24:11.413351+0000","last_fullsized":"2024-04-26T20:24:11.413351+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:49.172379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413585+0000","last_change":"2024-04-26T20:24:09.402853+0000","last_active":"2024-04-26T20:24:11.413585+0000","last_peered":"2024-04-26T20:24:11.413585+0000","last_clean":"2024-04-26T20:24:11.413585+0000","last_became_active":"2024-04-26T20:24:09.402559+0000","last_became_peered":"2024-04-26T20:24:09.402559+0000","last_unstale":"2024-04-26T20:24:11.413585+0000","last_undegraded":"2024-04-26T20:24:11.413585+0000","last_fullsized":"2024-04-26T20:24:11.413585+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:06.566389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611286+0000","last_change":"2024-04-26T20:24:10.409357+0000","last_active":"2024-04-26T20:24:11.611286+0000","last_peered":"2024-04-26T20:24:11.611286+0000","last_clean":"2024-04-26T20:24:11.611286+0000","last_became_active":"2024-04-26T20:24:10.409018+0000","last_became_peered":"2024-04-26T20:24:10.409018+0000","last_unstale":"2024-04-26T20:24:11.611286+0000","last_undegraded":"2024-04-26T20:24:11.611286+0000","last_fullsized":"2024-04-26T20:24:11.611286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:28.821951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412627+0000","last_change":"2024-04-26T20:24:09.407446+0000","last_active":"2024-04-26T20:24:11.412627+0000","last_peered":"2024-04-26T20:24:11.412627+0000","last_clean":"2024-04-26T20:24:11.412627+0000","last_became_active":"2024-04-26T20:24:09.407138+0000","last_became_peered":"2024-04-26T20:24:09.407138+0000","last_unstale":"2024-04-26T20:24:11.412627+0000","last_undegraded":"2024-04-26T20:24:11.412627+0000","last_fullsized":"2024-04-26T20:24:11.412627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:05.750310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039473+0000","last_change":"2024-04-26T20:24:10.416903+0000","last_active":"2024-04-26T20:24:12.039473+0000","last_peered":"2024-04-26T20:24:12.039473+0000","last_clean":"2024-04-26T20:24:12.039473+0000","last_became_active":"2024-04-26T20:24:10.416080+0000","last_became_peered":"2024-04-26T20:24:10.416080+0000","last_unstale":"2024-04-26T20:24:12.039473+0000","last_undegraded":"2024-04-26T20:24:12.039473+0000","last_fullsized":"2024-04-26T20:24:12.039473+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:57.904889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.411829+0000","last_change":"2024-04-26T20:24:09.406182+0000","last_active":"2024-04-26T20:24:11.411829+0000","last_peered":"2024-04-26T20:24:11.411829+0000","last_clean":"2024-04-26T20:24:11.411829+0000","last_became_active":"2024-04-26T20:24:09.405719+0000","last_became_peered":"2024-04-26T20:24:09.405719+0000","last_unstale":"2024-04-26T20:24:11.411829+0000","last_undegraded":"2024-04-26T20:24:11.411829+0000","last_fullsized":"2024-04-26T20:24:11.411829+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:45.757564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.411806+0000","last_change":"2024-04-26T20:24:10.417503+0000","last_active":"2024-04-26T20:24:11.411806+0000","last_peered":"2024-04-26T20:24:11.411806+0000","last_clean":"2024-04-26T20:24:11.411806+0000","last_became_active":"2024-04-26T20:24:10.417215+0000","last_became_peered":"2024-04-26T20:24:10.417215+0000","last_unstale":"2024-04-26T20:24:11.411806+0000","last_undegraded":"2024-04-26T20:24:11.411806+0000","last_fullsized":"2024-04-26T20:24:11.411806+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:35.112499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.537962+0000","last_change":"2024-04-26T20:24:09.407652+0000","last_active":"2024-04-26T20:24:11.537962+0000","last_peered":"2024-04-26T20:24:11.537962+0000","last_clean":"2024-04-26T20:24:11.537962+0000","last_became_active":"2024-04-26T20:24:09.407323+0000","last_became_peered":"2024-04-26T20:24:09.407323+0000","last_unstale":"2024-04-26T20:24:11.537962+0000","last_undegraded":"2024-04-26T20:24:11.537962+0000","last_fullsized":"2024-04-26T20:24:11.537962+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:34.472483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.040661+0000","last_change":"2024-04-26T20:24:10.418542+0000","last_active":"2024-04-26T20:24:12.040661+0000","last_peered":"2024-04-26T20:24:12.040661+0000","last_clean":"2024-04-26T20:24:12.040661+0000","last_became_active":"2024-04-26T20:24:10.418266+0000","last_became_peered":"2024-04-26T20:24:10.418266+0000","last_unstale":"2024-04-26T20:24:12.040661+0000","last_undegraded":"2024-04-26T20:24:12.040661+0000","last_fullsized":"2024-04-26T20:24:12.040661+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:21.585036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413271+0000","last_change":"2024-04-26T20:24:09.403967+0000","last_active":"2024-04-26T20:24:11.413271+0000","last_peered":"2024-04-26T20:24:11.413271+0000","last_clean":"2024-04-26T20:24:11.413271+0000","last_became_active":"2024-04-26T20:24:09.403658+0000","last_became_peered":"2024-04-26T20:24:09.403658+0000","last_unstale":"2024-04-26T20:24:11.413271+0000","last_undegraded":"2024-04-26T20:24:11.413271+0000","last_fullsized":"2024-04-26T20:24:11.413271+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:35.406719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412399+0000","last_change":"2024-04-26T20:24:10.407722+0000","last_active":"2024-04-26T20:24:11.412399+0000","last_peered":"2024-04-26T20:24:11.412399+0000","last_clean":"2024-04-26T20:24:11.412399+0000","last_became_active":"2024-04-26T20:24:10.407353+0000","last_became_peered":"2024-04-26T20:24:10.407353+0000","last_unstale":"2024-04-26T20:24:11.412399+0000","last_undegraded":"2024-04-26T20:24:11.412399+0000","last_fullsized":"2024-04-26T20:24:11.412399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:22.506318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538245+0000","last_change":"2024-04-26T20:24:09.408946+0000","last_active":"2024-04-26T20:24:11.538245+0000","last_peered":"2024-04-26T20:24:11.538245+0000","last_clean":"2024-04-26T20:24:11.538245+0000","last_became_active":"2024-04-26T20:24:09.408661+0000","last_became_peered":"2024-04-26T20:24:09.408661+0000","last_unstale":"2024-04-26T20:24:11.538245+0000","last_undegraded":"2024-04-26T20:24:11.538245+0000","last_fullsized":"2024-04-26T20:24:11.538245+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:29.072522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538176+0000","last_change":"2024-04-26T20:24:10.414873+0000","last_active":"2024-04-26T20:24:11.538176+0000","last_peered":"2024-04-26T20:24:11.538176+0000","last_clean":"2024-04-26T20:24:11.538176+0000","last_became_active":"2024-04-26T20:24:10.414602+0000","last_became_peered":"2024-04-26T20:24:10.414602+0000","last_unstale":"2024-04-26T20:24:11.538176+0000","last_undegraded":"2024-04-26T20:24:11.538176+0000","last_fullsized":"2024-04-26T20:24:11.538176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:53.099897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611177+0000","last_change":"2024-04-26T20:24:09.410720+0000","last_active":"2024-04-26T20:24:11.611177+0000","last_peered":"2024-04-26T20:24:11.611177+0000","last_clean":"2024-04-26T20:24:11.611177+0000","last_became_active":"2024-04-26T20:24:09.410423+0000","last_became_peered":"2024-04-26T20:24:09.410423+0000","last_unstale":"2024-04-26T20:24:11.611177+0000","last_undegraded":"2024-04-26T20:24:11.611177+0000","last_fullsized":"2024-04-26T20:24:11.611177+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:26.565983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413492+0000","last_change":"2024-04-26T20:24:10.415571+0000","last_active":"2024-04-26T20:24:11.413492+0000","last_peered":"2024-04-26T20:24:11.413492+0000","last_clean":"2024-04-26T20:24:11.413492+0000","last_became_active":"2024-04-26T20:24:10.415288+0000","last_became_peered":"2024-04-26T20:24:10.415288+0000","last_unstale":"2024-04-26T20:24:11.413492+0000","last_undegraded":"2024-04-26T20:24:11.413492+0000","last_fullsized":"2024-04-26T20:24:11.413492+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:16.157335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.538171+0000","last_change":"2024-04-26T20:24:09.401361+0000","last_active":"2024-04-26T20:24:11.538171+0000","last_peered":"2024-04-26T20:24:11.538171+0000","last_clean":"2024-04-26T20:24:11.538171+0000","last_became_active":"2024-04-26T20:24:09.401086+0000","last_became_peered":"2024-04-26T20:24:09.401086+0000","last_unstale":"2024-04-26T20:24:11.538171+0000","last_undegraded":"2024-04-26T20:24:11.538171+0000","last_fullsized":"2024-04-26T20:24:11.538171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:50.422477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413350+0000","last_change":"2024-04-26T20:24:10.415802+0000","last_active":"2024-04-26T20:24:11.413350+0000","last_peered":"2024-04-26T20:24:11.413350+0000","last_clean":"2024-04-26T20:24:11.413350+0000","last_became_active":"2024-04-26T20:24:10.415467+0000","last_became_peered":"2024-04-26T20:24:10.415467+0000","last_unstale":"2024-04-26T20:24:11.413350+0000","last_undegraded":"2024-04-26T20:24:11.413350+0000","last_fullsized":"2024-04-26T20:24:11.413350+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:50.472802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039643+0000","last_change":"2024-04-26T20:24:09.402277+0000","last_active":"2024-04-26T20:24:12.039643+0000","last_peered":"2024-04-26T20:24:12.039643+0000","last_clean":"2024-04-26T20:24:12.039643+0000","last_became_active":"2024-04-26T20:24:09.402003+0000","last_became_peered":"2024-04-26T20:24:09.402003+0000","last_unstale":"2024-04-26T20:24:12.039643+0000","last_undegraded":"2024-04-26T20:24:12.039643+0000","last_fullsized":"2024-04-26T20:24:12.039643+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:46.184964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412749+0000","last_change":"2024-04-26T20:24:10.413990+0000","last_active":"2024-04-26T20:24:11.412749+0000","last_peered":"2024-04-26T20:24:11.412749+0000","last_clean":"2024-04-26T20:24:11.412749+0000","last_became_active":"2024-04-26T20:24:10.413726+0000","last_became_peered":"2024-04-26T20:24:10.413726+0000","last_unstale":"2024-04-26T20:24:11.412749+0000","last_undegraded":"2024-04-26T20:24:11.412749+0000","last_fullsized":"2024-04-26T20:24:11.412749+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:09.962717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.611200+0000","last_change":"2024-04-26T20:24:09.401668+0000","last_active":"2024-04-26T20:24:11.611200+0000","last_peered":"2024-04-26T20:24:11.611200+0000","last_clean":"2024-04-26T20:24:11.611200+0000","last_became_active":"2024-04-26T20:24:09.401294+0000","last_became_peered":"2024-04-26T20:24:09.401294+0000","last_unstale":"2024-04-26T20:24:11.611200+0000","last_undegraded":"2024-04-26T20:24:11.611200+0000","last_fullsized":"2024-04-26T20:24:11.611200+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:20.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:12.039605+0000","last_change":"2024-04-26T20:24:10.417123+0000","last_active":"2024-04-26T20:24:12.039605+0000","last_peered":"2024-04-26T20:24:12.039605+0000","last_clean":"2024-04-26T20:24:12.039605+0000","last_became_active":"2024-04-26T20:24:10.416828+0000","last_became_peered":"2024-04-26T20:24:10.416828+0000","last_unstale":"2024-04-26T20:24:12.039605+0000","last_undegraded":"2024-04-26T20:24:12.039605+0000","last_fullsized":"2024-04-26T20:24:12.039605+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:36.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412393+0000","last_change":"2024-04-26T20:24:09.412877+0000","last_active":"2024-04-26T20:24:11.412393+0000","last_peered":"2024-04-26T20:24:11.412393+0000","last_clean":"2024-04-26T20:24:11.412393+0000","last_became_active":"2024-04-26T20:24:09.412615+0000","last_became_peered":"2024-04-26T20:24:09.412615+0000","last_unstale":"2024-04-26T20:24:11.412393+0000","last_undegraded":"2024-04-26T20:24:11.412393+0000","last_fullsized":"2024-04-26T20:24:11.412393+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:02.144918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.412604+0000","last_change":"2024-04-26T20:24:10.411821+0000","last_active":"2024-04-26T20:24:11.412604+0000","last_peered":"2024-04-26T20:24:11.412604+0000","last_clean":"2024-04-26T20:24:11.412604+0000","last_became_active":"2024-04-26T20:24:10.411507+0000","last_became_peered":"2024-04-26T20:24:10.411507+0000","last_unstale":"2024-04-26T20:24:11.412604+0000","last_undegraded":"2024-04-26T20:24:11.412604+0000","last_fullsized":"2024-04-26T20:24:11.412604+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:09.382705+0000","last_clean_scrub_stamp":"2024-04-26T20:24:09.382705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:36.412020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.413112+0000","last_change":"2024-04-26T20:24:09.407563+0000","last_active":"2024-04-26T20:24:11.413112+0000","last_peered":"2024-04-26T20:24:11.413112+0000","last_clean":"2024-04-26T20:24:11.413112+0000","last_became_active":"2024-04-26T20:24:09.407184+0000","last_became_peered":"2024-04-26T20:24:09.407184+0000","last_unstale":"2024-04-26T20:24:11.413112+0000","last_undegraded":"2024-04-26T20:24:11.413112+0000","last_fullsized":"2024-04-26T20:24:11.413112+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:24:08.377404+0000","last_clean_scrub_stamp":"2024-04-26T20:24:08.377404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:43.407221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":219,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T20:24:11.537742+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:24:11.537742+0000","last_peered":"2024-04-26T20:24:11.537742+0000","last_clean":"2024-04-26T20:24:11.537742+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:24:11.537742+0000","last_undegraded":"2024-04-26T20:24:11.537742+0000","last_fullsized":"2024-04-26T20:24:11.537742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":806315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":216529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":222227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":540672,"data_stored":217469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":552960,"data_stored":222795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":217491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":222227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":812559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:24:13.779 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:24:13.779 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T20:24:13.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.922+0000 7fe73872b640 1 Processor -- start 2024-04-26T20:24:13.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.922+0000 7fe73872b640 1 -- start start 2024-04-26T20:24:13.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.923+0000 7fe73872b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe7301609e0 0x7fe73006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.923+0000 7fe73872b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe730072400 con 0x7fe7300ae1f0 2024-04-26T20:24:13.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.923+0000 7fe73872b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe730073de0 con 0x7fe730071f70 2024-04-26T20:24:13.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.923+0000 7fe73872b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe7300add80 con 0x7fe7301609e0 2024-04-26T20:24:13.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.923+0000 7fe736cc8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe7300ae1f0 0x7fe7300ae640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59734/0 (socket says 172.21.15.37:59734) 2024-04-26T20:24:13.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.923+0000 7fe736cc8640 1 -- 172.21.15.37:0/2817904359 learned_addr learned my addr 172.21.15.37:0/2817904359 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:13.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.923+0000 7fe7364c7640 1 --2- 172.21.15.37:0/2817904359 >> v2:172.21.15.134:3300/0 conn(0x7fe7301609e0 0x7fe73006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.923+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969378885 0 0) 0x7fe730072400 con 0x7fe7300ae1f0 2024-04-26T20:24:13.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.923+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe718003520 con 0x7fe7300ae1f0 2024-04-26T20:24:13.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2313482533 0 0) 0x7fe718003520 con 0x7fe7300ae1f0 2024-04-26T20:24:13.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 >> v2:172.21.15.134:3300/0 conn(0x7fe7301609e0 msgr2=0x7fe73006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:13.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 --2- 172.21.15.37:0/2817904359 >> v2:172.21.15.134:3300/0 conn(0x7fe7301609e0 0x7fe73006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 >> v1:172.21.15.134:6789/0 conn(0x7fe730071f70 legacy=0x7fe73006e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7300740c0 con 0x7fe7300ae1f0 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fe7200019f0 con 0x7fe7300ae1f0 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 >> v1:172.21.15.37:6789/0 conn(0x7fe7300ae1f0 legacy=0x7fe7300ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 --2- 172.21.15.37:0/2817904359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7180039b0 0x7fe718003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 --2- 172.21.15.37:0/2817904359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe718004800 0x7fe718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 --2- 172.21.15.37:0/2817904359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7180051c0 0x7fe71800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe71800c130 con 0x7fe7180039b0 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe71800c2e0 con 0x7fe7180051c0 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe71800c4e0 con 0x7fe718004800 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7364c7640 1 --2- 172.21.15.37:0/2817904359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7180039b0 0x7fe718003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe736cc8640 1 --2- 172.21.15.37:0/2817904359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7180051c0 0x7fe71800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe735cc6640 1 --2- 172.21.15.37:0/2817904359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe718004800 0x7fe718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7364c7640 1 -- 172.21.15.37:0/2817904359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe718004800 msgr2=0x7fe718004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7364c7640 1 --2- 172.21.15.37:0/2817904359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe718004800 0x7fe718004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7364c7640 1 -- 172.21.15.37:0/2817904359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7180051c0 msgr2=0x7fe71800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7364c7640 1 --2- 172.21.15.37:0/2817904359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7180051c0 0x7fe71800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe736cc8640 1 -- 172.21.15.37:0/2817904359 reap_dead start 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.924+0000 7fe7364c7640 1 -- 172.21.15.37:0/2817904359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe718003520 con 0x7fe7180039b0 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.925+0000 7fe7364c7640 1 --2- 172.21.15.37:0/2817904359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7180039b0 0x7fe718003dd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe728009ac0 tx=0x7fe718005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5235ad84bb50925f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.925+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728036070 con 0x7fe7180039b0 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.925+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe728031400 con 0x7fe7180039b0 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.925+0000 7fe7354c5640 1 -- 172.21.15.37:0/2817904359 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728009690 con 0x7fe7180039b0 2024-04-26T20:24:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.925+0000 7fe73872b640 1 -- 172.21.15.37:0/2817904359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7180039b0 msgr2=0x7fe718003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.925+0000 7fe73872b640 1 --2- 172.21.15.37:0/2817904359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7180039b0 0x7fe718003dd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe728009ac0 tx=0x7fe718005830 comp rx=0 tx=0).stop 2024-04-26T20:24:13.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.925+0000 7fe73872b640 1 -- 172.21.15.37:0/2817904359 shutdown_connections 2024-04-26T20:24:13.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.925+0000 7fe73872b640 1 --2- 172.21.15.37:0/2817904359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7180039b0 0x7fe718003dd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.925+0000 7fe73872b640 1 -- 172.21.15.37:0/2817904359 >> 172.21.15.37:0/2817904359 conn(0x7fe7300d0d70 msgr2=0x7fe7300d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:13.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.926+0000 7fe73872b640 1 -- 172.21.15.37:0/2817904359 shutdown_connections 2024-04-26T20:24:13.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.926+0000 7fe73872b640 1 -- 172.21.15.37:0/2817904359 wait complete. 2024-04-26T20:24:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.927+0000 7fe73872b640 1 Processor -- start 2024-04-26T20:24:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.927+0000 7fe73872b640 1 -- start start 2024-04-26T20:24:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.927+0000 7fe73872b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7180039b0 0x7fe73014c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.927+0000 7fe73872b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe73009dce0 0x7fe73009e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.927+0000 7fe73872b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7300c6850 0x7fe7300c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.927+0000 7fe73872b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe7300d1c50 con 0x7fe73009dce0 2024-04-26T20:24:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.927+0000 7fe73872b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7300d1ad0 con 0x7fe7180039b0 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.927+0000 7fe73872b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe73006dcf0 con 0x7fe7300c6850 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe735cc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe73009dce0 0x7fe73009e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe735cc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe73009dce0 0x7fe73009e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43938/0 (socket says 172.21.15.37:43938) 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe735cc6640 1 -- 172.21.15.37:0/2228739745 learned_addr learned my addr 172.21.15.37:0/2228739745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe7364c7640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7180039b0 0x7fe73014c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe736cc8640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7300c6850 0x7fe7300c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe735cc6640 1 -- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7300c6850 msgr2=0x7fe7300c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe735cc6640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7300c6850 0x7fe7300c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe735cc6640 1 -- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7180039b0 msgr2=0x7fe73014c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe735cc6640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7180039b0 0x7fe73014c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe735cc6640 1 -- 172.21.15.37:0/2228739745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7301e7fa0 con 0x7fe73009dce0 2024-04-26T20:24:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe7364c7640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7180039b0 0x7fe73014c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe735cc6640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe73009dce0 0x7fe73009e140 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe72400e530 tx=0x7fe72400e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe72effd640 1 -- 172.21.15.37:0/2228739745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7240053a0 con 0x7fe73009dce0 2024-04-26T20:24:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe72effd640 1 -- 172.21.15.37:0/2228739745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe724018070 con 0x7fe73009dce0 2024-04-26T20:24:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe72effd640 1 -- 172.21.15.37:0/2228739745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724003810 con 0x7fe73009dce0 2024-04-26T20:24:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe73872b640 1 -- 172.21.15.37:0/2228739745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7301e8280 con 0x7fe73009dce0 2024-04-26T20:24:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.928+0000 7fe73872b640 1 -- 172.21.15.37:0/2228739745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7301e8840 con 0x7fe73009dce0 2024-04-26T20:24:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.929+0000 7fe73872b640 1 -- 172.21.15.37:0/2228739745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7300b2670 con 0x7fe73009dce0 2024-04-26T20:24:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.930+0000 7fe72effd640 1 -- 172.21.15.37:0/2228739745 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fe7240039b0 con 0x7fe73009dce0 2024-04-26T20:24:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.930+0000 7fe72effd640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe71c06cbf0 0x7fe71c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.930+0000 7fe72effd640 1 -- 172.21.15.37:0/2228739745 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe72408bc90 con 0x7fe73009dce0 2024-04-26T20:24:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.930+0000 7fe7364c7640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe71c06cbf0 0x7fe71c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.931+0000 7fe7364c7640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe71c06cbf0 0x7fe71c06f0b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe728033000 tx=0x7fe728008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:13.934+0000 7fe72effd640 1 -- 172.21.15.37:0/2228739745 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe7300b2670 con 0x7fe73009dce0 2024-04-26T20:24:14.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.191+0000 7fe73872b640 1 -- 172.21.15.37:0/2228739745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe73006a360 con 0x7fe73009dce0 2024-04-26T20:24:14.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.193+0000 7fe72effd640 1 -- 172.21.15.37:0/2228739745 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fe724091020 con 0x7fe73009dce0 2024-04-26T20:24:14.194 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-26T20:24:14.194 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCkDCxmQTcnCBAAFh1Ew2ihHFtREYikzwF+Dw== 2024-04-26T20:24:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-04-26T20:24:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-04-26T20:24:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T20:24:14.195 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-04-26T20:24:14.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 -- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe71c06cbf0 msgr2=0x7fe71c06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:14.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe71c06cbf0 0x7fe71c06f0b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe728033000 tx=0x7fe728008030 comp rx=0 tx=0).stop 2024-04-26T20:24:14.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 -- 172.21.15.37:0/2228739745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe73009dce0 msgr2=0x7fe73009e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:14.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe73009dce0 0x7fe73009e140 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe72400e530 tx=0x7fe72400e9f0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 -- 172.21.15.37:0/2228739745 shutdown_connections 2024-04-26T20:24:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe71c06cbf0 0x7fe71c06f0b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7300c6850 0x7fe7300c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe73009dce0 0x7fe73009e140 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 --2- 172.21.15.37:0/2228739745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7180039b0 0x7fe73014c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 -- 172.21.15.37:0/2228739745 >> 172.21.15.37:0/2228739745 conn(0x7fe7300d0d70 msgr2=0x7fe7300cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 -- 172.21.15.37:0/2228739745 shutdown_connections 2024-04-26T20:24:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.196+0000 7fe73872b640 1 -- 172.21.15.37:0/2228739745 wait complete. 2024-04-26T20:24:14.208 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:24:14.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.307+0000 7f9a8622a640 1 Processor -- start 2024-04-26T20:24:14.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.307+0000 7f9a8622a640 1 -- start start 2024-04-26T20:24:14.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.308+0000 7f9a8622a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9a80159a90 0x7f9a8006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.308+0000 7f9a8622a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80070130 0x7f9a8015fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.308+0000 7f9a8622a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a80072e30 con 0x7f9a80160150 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.308+0000 7f9a8622a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f9a80160580 con 0x7f9a80159a90 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.308+0000 7f9a8622a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9a80073110 con 0x7f9a80070130 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.308+0000 7f9a7f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9a80159a90 0x7f9a8006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.308+0000 7f9a7ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9a80160150 0x7f9a8015c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59738/0 (socket says 172.21.15.37:59738) 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.308+0000 7f9a7ffff640 1 -- 172.21.15.37:0/2005676814 learned_addr learned my addr 172.21.15.37:0/2005676814 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.308+0000 7f9a7effd640 1 --2- 172.21.15.37:0/2005676814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80070130 0x7f9a8015fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.309+0000 7f9a7f7fe640 1 -- 172.21.15.37:0/2005676814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80070130 msgr2=0x7f9a8015fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.309+0000 7f9a7f7fe640 1 --2- 172.21.15.37:0/2005676814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80070130 0x7f9a8015fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.309+0000 7f9a7f7fe640 1 -- 172.21.15.37:0/2005676814 >> v1:172.21.15.37:6789/0 conn(0x7f9a80160150 legacy=0x7f9a8015c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.309+0000 7f9a7e7fc640 1 -- 172.21.15.37:0/2005676814 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982320780 0 0) 0x7f9a80072e30 con 0x7f9a80160150 2024-04-26T20:24:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.309+0000 7f9a7f7fe640 1 -- 172.21.15.37:0/2005676814 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a800733f0 con 0x7f9a80159a90 2024-04-26T20:24:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.309+0000 7f9a7f7fe640 1 --2- 172.21.15.37:0/2005676814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a80159a90 0x7f9a8006fbf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9a700097a0 tx=0x7f9a7002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c696fe01aba38d20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.309+0000 7f9a7e7fc640 1 -- 172.21.15.37:0/2005676814 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70037070 con 0x7f9a80159a90 2024-04-26T20:24:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.309+0000 7f9a7e7fc640 1 -- 172.21.15.37:0/2005676814 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7002adb0 con 0x7f9a80159a90 2024-04-26T20:24:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.310+0000 7f9a7e7fc640 1 -- 172.21.15.37:0/2005676814 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70032660 con 0x7f9a80159a90 2024-04-26T20:24:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.310+0000 7f9a8622a640 1 -- 172.21.15.37:0/2005676814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a80159a90 msgr2=0x7f9a8006fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.310+0000 7f9a8622a640 1 --2- 172.21.15.37:0/2005676814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a80159a90 0x7f9a8006fbf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9a700097a0 tx=0x7f9a7002a060 comp rx=0 tx=0).stop 2024-04-26T20:24:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.310+0000 7f9a8622a640 1 -- 172.21.15.37:0/2005676814 shutdown_connections 2024-04-26T20:24:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.310+0000 7f9a8622a640 1 --2- 172.21.15.37:0/2005676814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80070130 0x7f9a8015fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.310+0000 7f9a8622a640 1 --2- 172.21.15.37:0/2005676814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a80159a90 0x7f9a8006fbf0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.310+0000 7f9a8622a640 1 -- 172.21.15.37:0/2005676814 >> 172.21.15.37:0/2005676814 conn(0x7f9a800d0d50 msgr2=0x7f9a800d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.310+0000 7f9a8622a640 1 -- 172.21.15.37:0/2005676814 shutdown_connections 2024-04-26T20:24:14.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.310+0000 7f9a8622a640 1 -- 172.21.15.37:0/2005676814 wait complete. 2024-04-26T20:24:14.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.311+0000 7f9a8622a640 1 Processor -- start 2024-04-26T20:24:14.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.311+0000 7f9a8622a640 1 -- start start 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a8622a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80070130 0x7f9a80147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a8622a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80159a90 0x7f9a8013f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a8622a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a80160150 0x7f9a80143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a8622a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9a80159e70 con 0x7f9a80160150 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a8622a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801609d0 con 0x7f9a80070130 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a8622a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9a80159ff0 con 0x7f9a80159a90 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a7ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a80160150 0x7f9a80143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a7ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a80160150 0x7f9a80143a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43968/0 (socket says 172.21.15.37:43968) 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a7ffff640 1 -- 172.21.15.37:0/3169646265 learned_addr learned my addr 172.21.15.37:0/3169646265 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a7effd640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80159a90 0x7f9a8013f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a7f7fe640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80070130 0x7f9a80147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a7ffff640 1 -- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80159a90 msgr2=0x7f9a8013f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a7ffff640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80159a90 0x7f9a8013f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a7ffff640 1 -- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80070130 msgr2=0x7f9a80147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a7ffff640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80070130 0x7f9a80147dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a7ffff640 1 -- 172.21.15.37:0/3169646265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8013f850 con 0x7f9a80160150 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.312+0000 7f9a7f7fe640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80070130 0x7f9a80147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.313+0000 7f9a7ffff640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a80160150 0x7f9a80143a30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800c7a0 tx=0x7f9a7800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.313+0000 7f9a67fff640 1 -- 172.21.15.37:0/3169646265 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78018070 con 0x7f9a80160150 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.313+0000 7f9a67fff640 1 -- 172.21.15.37:0/3169646265 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7800a640 con 0x7f9a80160150 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.313+0000 7f9a8622a640 1 -- 172.21.15.37:0/3169646265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a801446f0 con 0x7f9a80160150 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.313+0000 7f9a67fff640 1 -- 172.21.15.37:0/3169646265 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7800aab0 con 0x7f9a80160150 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.313+0000 7f9a8622a640 1 -- 172.21.15.37:0/3169646265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a80144cb0 con 0x7f9a80160150 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.314+0000 7f9a8622a640 1 -- 172.21.15.37:0/3169646265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8006a360 con 0x7f9a80160150 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.314+0000 7f9a67fff640 1 -- 172.21.15.37:0/3169646265 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f9a7801b070 con 0x7f9a80160150 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.315+0000 7f9a67fff640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9a6c06cbc0 0x7f9a6c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.315+0000 7f9a67fff640 1 -- 172.21.15.37:0/3169646265 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f9a7808b6d0 con 0x7f9a80160150 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.315+0000 7f9a7f7fe640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9a6c06cbc0 0x7f9a6c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.315+0000 7f9a7f7fe640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9a6c06cbc0 0x7f9a6c06f080 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9a70034000 tx=0x7f9a7002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:14.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.319+0000 7f9a67fff640 1 -- 172.21.15.37:0/3169646265 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a780596d0 con 0x7f9a80160150 2024-04-26T20:24:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.544+0000 7f9a8622a640 1 -- 172.21.15.37:0/3169646265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a80150410 con 0x7f9a80160150 2024-04-26T20:24:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.544+0000 7f9a67fff640 1 -- 172.21.15.37:0/3169646265 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5178 (secure 0 0 0) 0x7f9a78058fd0 con 0x7f9a80160150 2024-04-26T20:24:14.546 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:24:14.546 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/689917778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":689917778},{"type":"v1","addr":"172.21.15.134:6836","nonce":689917778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5362,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/497972484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":497972484},{"type":"v1","addr":"172.21.15.134:6837","nonce":497972484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/221502372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":221502372},{"type":"v1","addr":"172.21.15.37:6833","nonce":221502372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:24:09.772695+0000","modified":"2024-04-26T20:24:11.765632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1915220099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1915220099},{"type":"v1","addr":"172.21.15.37:6835","nonce":1915220099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5688,"qdb_cluster":[5688]},"id":3}]} 2024-04-26T20:24:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 -- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9a6c06cbc0 msgr2=0x7f9a6c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9a6c06cbc0 0x7f9a6c06f080 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9a70034000 tx=0x7f9a7002e000 comp rx=0 tx=0).stop 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 -- 172.21.15.37:0/3169646265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a80160150 msgr2=0x7f9a80143a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a80160150 0x7f9a80143a30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800c7a0 tx=0x7f9a7800cc60 comp rx=0 tx=0).stop 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 -- 172.21.15.37:0/3169646265 shutdown_connections 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a80160150 0x7f9a80143a30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9a6c06cbc0 0x7f9a6c06f080 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80159a90 0x7f9a8013f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 --2- 172.21.15.37:0/3169646265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80070130 0x7f9a80147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 -- 172.21.15.37:0/3169646265 >> 172.21.15.37:0/3169646265 conn(0x7f9a800d0d50 msgr2=0x7f9a801586f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 -- 172.21.15.37:0/3169646265 shutdown_connections 2024-04-26T20:24:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:14.546+0000 7f9a8622a640 1 -- 172.21.15.37:0/3169646265 wait complete. 2024-04-26T20:24:14.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:24:09.772695+0000', 'modified': '2024-04-26T20:24:11.765632+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5688}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5688': {'gid': 5688, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/1915220099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 1915220099}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1915220099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5688, 'qdb_cluster': [5688]} 2024-04-26T20:24:14.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:24:14.962 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T20:24:15.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.070+0000 7f1be9ec4640 1 Processor -- start 2024-04-26T20:24:15.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.070+0000 7f1be9ec4640 1 -- start start 2024-04-26T20:24:15.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be9ec4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1be415f380 0x7f1be40d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be9ec4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1be415f760 0x7f1be40d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be9ec4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1be40d5b50 0x7f1be40cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be9ec4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f1be40d43a0 con 0x7f1be415f760 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be9ec4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1be40d3de0 con 0x7f1be40d5b50 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be9ec4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1be40d40c0 con 0x7f1be415f380 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1be415f760 0x7f1be40d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1be415f760 0x7f1be40d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43978/0 (socket says 172.21.15.37:43978) 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be2ffd640 1 -- 172.21.15.37:0/3834215262 learned_addr learned my addr 172.21.15.37:0/3834215262 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be37fe640 1 --2- 172.21.15.37:0/3834215262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1be415f380 0x7f1be40d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be3fff640 1 --2- 172.21.15.37:0/3834215262 >> v2:172.21.15.134:3300/0 conn(0x7f1be40d5b50 0x7f1be40cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be2ffd640 1 -- 172.21.15.37:0/3834215262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1be415f380 msgr2=0x7f1be40d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/3834215262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1be415f380 0x7f1be40d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be2ffd640 1 -- 172.21.15.37:0/3834215262 >> v2:172.21.15.134:3300/0 conn(0x7f1be40d5b50 msgr2=0x7f1be40cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/3834215262 >> v2:172.21.15.134:3300/0 conn(0x7f1be40d5b50 0x7f1be40cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.071+0000 7f1be2ffd640 1 -- 172.21.15.37:0/3834215262 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be40b8280 con 0x7f1be415f760 2024-04-26T20:24:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.072+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/3834215262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be415f760 0x7f1be40d2110 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1be408d560 tx=0x7f1bd802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df69db85c033a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.072+0000 7f1be27fc640 1 -- 172.21.15.37:0/3834215262 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8037070 con 0x7f1be415f760 2024-04-26T20:24:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.072+0000 7f1be27fc640 1 -- 172.21.15.37:0/3834215262 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd8032440 con 0x7f1be415f760 2024-04-26T20:24:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.072+0000 7f1be27fc640 1 -- 172.21.15.37:0/3834215262 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8032810 con 0x7f1be415f760 2024-04-26T20:24:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.073+0000 7f1be9ec4640 1 -- 172.21.15.37:0/3834215262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be415f760 msgr2=0x7f1be40d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.073+0000 7f1be9ec4640 1 --2- 172.21.15.37:0/3834215262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be415f760 0x7f1be40d2110 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1be408d560 tx=0x7f1bd802a270 comp rx=0 tx=0).stop 2024-04-26T20:24:15.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.073+0000 7f1be9ec4640 1 -- 172.21.15.37:0/3834215262 shutdown_connections 2024-04-26T20:24:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.073+0000 7f1be9ec4640 1 --2- 172.21.15.37:0/3834215262 >> v2:172.21.15.134:3300/0 conn(0x7f1be40d5b50 0x7f1be40cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.073+0000 7f1be9ec4640 1 --2- 172.21.15.37:0/3834215262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be415f760 0x7f1be40d2110 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.073+0000 7f1be9ec4640 1 --2- 172.21.15.37:0/3834215262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1be415f380 0x7f1be40d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.073+0000 7f1be9ec4640 1 -- 172.21.15.37:0/3834215262 >> 172.21.15.37:0/3834215262 conn(0x7f1be409e560 msgr2=0x7f1be40d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.073+0000 7f1be9ec4640 1 -- 172.21.15.37:0/3834215262 shutdown_connections 2024-04-26T20:24:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.073+0000 7f1be9ec4640 1 -- 172.21.15.37:0/3834215262 wait complete. 2024-04-26T20:24:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.074+0000 7f1be9ec4640 1 Processor -- start 2024-04-26T20:24:15.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.074+0000 7f1be9ec4640 1 -- start start 2024-04-26T20:24:15.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.075+0000 7f1be9ec4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be40d5b50 0x7f1be40cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.075+0000 7f1be9ec4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1be415f380 0x7f1be414dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.075+0000 7f1be9ec4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be415f760 0x7f1be41f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.075+0000 7f1be9ec4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1be40d1280 con 0x7f1be415f760 2024-04-26T20:24:15.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.075+0000 7f1be9ec4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1be40d1100 con 0x7f1be40d5b50 2024-04-26T20:24:15.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.075+0000 7f1be9ec4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1be40d3840 con 0x7f1be415f380 2024-04-26T20:24:15.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.075+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be415f760 0x7f1be41f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.075+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be415f760 0x7f1be41f8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43984/0 (socket says 172.21.15.37:43984) 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.075+0000 7f1be3fff640 1 -- 172.21.15.37:0/2355714831 learned_addr learned my addr 172.21.15.37:0/2355714831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.075+0000 7f1be37fe640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be40d5b50 0x7f1be40cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.075+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1be415f380 0x7f1be414dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1be3fff640 1 -- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1be415f380 msgr2=0x7f1be414dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1be3fff640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1be415f380 0x7f1be414dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1be3fff640 1 -- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be40d5b50 msgr2=0x7f1be40cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1be3fff640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be40d5b50 0x7f1be40cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1be3fff640 1 -- 172.21.15.37:0/2355714831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be40cdab0 con 0x7f1be415f760 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1be415f380 0x7f1be414dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1be37fe640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be40d5b50 0x7f1be40cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1be3fff640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be415f760 0x7f1be41f8550 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc007e90 tx=0x7f1bdc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1bcbfff640 1 -- 172.21.15.37:0/2355714831 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc014420 con 0x7f1be415f760 2024-04-26T20:24:15.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1bcbfff640 1 -- 172.21.15.37:0/2355714831 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bdc019070 con 0x7f1be415f760 2024-04-26T20:24:15.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1bcbfff640 1 -- 172.21.15.37:0/2355714831 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc014820 con 0x7f1be415f760 2024-04-26T20:24:15.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1be9ec4640 1 -- 172.21.15.37:0/2355714831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be40cdd90 con 0x7f1be415f760 2024-04-26T20:24:15.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.076+0000 7f1be9ec4640 1 -- 172.21.15.37:0/2355714831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be40ce350 con 0x7f1be415f760 2024-04-26T20:24:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.078+0000 7f1bcbfff640 1 -- 172.21.15.37:0/2355714831 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f1bdc0149c0 con 0x7f1be415f760 2024-04-26T20:24:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.078+0000 7f1bcbfff640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1bd006cbc0 0x7f1bd006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.078+0000 7f1bcbfff640 1 -- 172.21.15.37:0/2355714831 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1bdc08c830 con 0x7f1be415f760 2024-04-26T20:24:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.078+0000 7f1be37fe640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1bd006cbc0 0x7f1bd006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.079+0000 7f1becbd9740 1 Processor -- start 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.079+0000 7f1becbd9740 1 -- start start 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.079+0000 7f1be37fe640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1bd006cbc0 0x7f1bd006f080 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4006fd0 tx=0x7f1bd4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1dc8440 0x5587f1dc8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1de5ce0 0x5587f1de6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1d9e120 0x5587f1de43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1becbd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5587f1d02740 con 0x5587f1d9e120 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1becbd9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587f1c9e3a0 con 0x5587f1dc8440 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1becbd9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5587f1cfe700 con 0x5587f1de5ce0 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1d9e120 0x5587f1de43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1d9e120 0x5587f1de43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43990/0 (socket says 172.21.15.37:43990) 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1be3fff640 1 -- 172.21.15.37:0/1726688779 learned_addr learned my addr 172.21.15.37:0/1726688779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/1726688779 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1de5ce0 0x5587f1de6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1be37fe640 1 --2- 172.21.15.37:0/1726688779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1dc8440 0x5587f1dc8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1be3fff640 1 -- 172.21.15.37:0/1726688779 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1de5ce0 msgr2=0x5587f1de6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1be3fff640 1 --2- 172.21.15.37:0/1726688779 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1de5ce0 0x5587f1de6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1be3fff640 1 -- 172.21.15.37:0/1726688779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1dc8440 msgr2=0x5587f1dc8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1be3fff640 1 --2- 172.21.15.37:0/1726688779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1dc8440 0x5587f1dc8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1be3fff640 1 -- 172.21.15.37:0/1726688779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587f1cba750 con 0x5587f1d9e120 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.080+0000 7f1be3fff640 1 --2- 172.21.15.37:0/1726688779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1d9e120 0x5587f1de43a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc08b0d0 tx=0x7f1bdc05a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4255498ed182953 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1bc97fa640 1 -- 172.21.15.37:0/1726688779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc062df0 con 0x5587f1d9e120 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1bc97fa640 1 -- 172.21.15.37:0/1726688779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bdc08c040 con 0x5587f1d9e120 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1bc97fa640 1 -- 172.21.15.37:0/1726688779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc056880 con 0x5587f1d9e120 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1becbd9740 1 -- 172.21.15.37:0/1726688779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1d9e120 msgr2=0x5587f1de43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1726688779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1d9e120 0x5587f1de43a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc08b0d0 tx=0x7f1bdc05a910 comp rx=0 tx=0).stop 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1becbd9740 1 -- 172.21.15.37:0/1726688779 shutdown_connections 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1726688779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1d9e120 0x5587f1de43a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1726688779 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1de5ce0 0x5587f1de6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1726688779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1dc8440 0x5587f1dc8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1becbd9740 1 -- 172.21.15.37:0/1726688779 >> 172.21.15.37:0/1726688779 conn(0x5587f1d9aa50 msgr2=0x5587f1de3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:15.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1becbd9740 1 -- 172.21.15.37:0/1726688779 shutdown_connections 2024-04-26T20:24:15.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.081+0000 7f1becbd9740 1 -- 172.21.15.37:0/1726688779 wait complete. 2024-04-26T20:24:15.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.082+0000 7f1becbd9740 1 -- start start 2024-04-26T20:24:15.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.082+0000 7f1becbd9740 1 Processor -- start 2024-04-26T20:24:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1dc8440 0x5587f1df3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1de5ce0 0x5587f1df3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1df3fb0 0x5587f1d976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1becbd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5587f1d9eec0 con 0x5587f1de5ce0 2024-04-26T20:24:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1becbd9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587f1c89fe0 con 0x5587f1dc8440 2024-04-26T20:24:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1becbd9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5587f1d98980 con 0x5587f1df3fb0 2024-04-26T20:24:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1becbd9740 10 client.0 _ll_register_callbacks cb 0x5587f1ca0150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1be2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1de5ce0 0x5587f1df3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1be2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1de5ce0 0x5587f1df3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44002/0 (socket says 172.21.15.37:44002) 2024-04-26T20:24:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1be2ffd640 1 -- 172.21.15.37:0/3502095514 learned_addr learned my addr 172.21.15.37:0/3502095514 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1be3fff640 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1df3fb0 0x5587f1d976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1be37fe640 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1dc8440 0x5587f1df3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1be2ffd640 1 -- 172.21.15.37:0/3502095514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1df3fb0 msgr2=0x5587f1d976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1df3fb0 0x5587f1d976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1be2ffd640 1 -- 172.21.15.37:0/3502095514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1dc8440 msgr2=0x5587f1df3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1dc8440 0x5587f1df3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1be2ffd640 1 -- 172.21.15.37:0/3502095514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587f1a142d0 con 0x5587f1de5ce0 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1de5ce0 0x5587f1df3a70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1bd802a2b0 tx=0x7f1bd8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b95c0061a9940d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.083+0000 7f1bc27fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1bc27fc640 1 -- 172.21.15.37:0/3502095514 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8037040 con 0x5587f1de5ce0 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1bc27fc640 1 -- 172.21.15.37:0/3502095514 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1bd8047aa0 con 0x5587f1de5ce0 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1becbd9740 1 -- 172.21.15.37:0/3502095514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bdc08c040 con 0x5587f1de5ce0 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1bc27fc640 1 -- 172.21.15.37:0/3502095514 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1bd8047ca0 con 0x5587f1de5ce0 2024-04-26T20:24:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1becbd9740 10 client.5844 fetch_fsmap learned FSMap version 23 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1becbd9740 1 -- 172.21.15.37:0/3502095514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5587f1ade6b0 con 0x5587f1de5ce0 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1bc27fc640 1 -- 172.21.15.37:0/3502095514 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bdc08c040 con 0x5587f1de5ce0 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1bc27fc640 1 -- 172.21.15.37:0/3502095514 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f1bd804e050 con 0x5587f1de5ce0 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1becbd9740 10 client.5844 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1becbd9740 10 client.5844 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1becbd9740 20 client.5844 populate_metadata read hostname 'smithi037' 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1becbd9740 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.37:6834/1915220099,v1:172.21.15.37:6835/1915220099] conn(0x5587f1dc69a0 0x5587f1dc6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1becbd9740 4 client.5844 mds_command: new command op to 5688 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1becbd9740 1 -- 172.21.15.37:0/3502095514 --> [v2:172.21.15.37:6834/1915220099,v1:172.21.15.37:6835/1915220099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587f1d10350 con 0x5587f1dc69a0 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.084+0000 7f1be37fe640 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.37:6834/1915220099,v1:172.21.15.37:6835/1915220099] conn(0x5587f1dc69a0 0x5587f1dc6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1be37fe640 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.37:6834/1915220099,v1:172.21.15.37:6835/1915220099] conn(0x5587f1dc69a0 0x5587f1dc6d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39eb9dcfcfddccd6 server_cookie=899f471dc970ffbc in_seq=0 out_seq=0 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1bc27fc640 10 client.5844 ms_handle_connect on v2:172.21.15.37:6834/1915220099 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1bc27fc640 1 -- 172.21.15.37:0/3502095514 <== mds.0 v2:172.21.15.37:6834/1915220099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5587f1d10350 con 0x5587f1dc69a0 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1bc27fc640 10 client.5844 handle_command_reply: tid=0 2024-04-26T20:24:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1becbd9740 1 client.5844 shutdown 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1becbd9740 1 -- 172.21.15.37:0/3502095514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1de5ce0 msgr2=0x5587f1df3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1becbd9740 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1de5ce0 0x5587f1df3a70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1bd802a2b0 tx=0x7f1bd8004730 comp rx=0 tx=0).stop 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1becbd9740 1 -- 172.21.15.37:0/3502095514 shutdown_connections 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1becbd9740 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.37:6834/1915220099,v1:172.21.15.37:6835/1915220099] conn(0x5587f1dc69a0 0x5587f1dc6d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1becbd9740 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1df3fb0 0x5587f1d976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1becbd9740 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1de5ce0 0x5587f1df3a70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1bc27fc640 0 client.5844 ms_handle_reset on v2:172.21.15.37:6834/1915220099 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1becbd9740 1 --2- 172.21.15.37:0/3502095514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1dc8440 0x5587f1df3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.085+0000 7f1becbd9740 1 -- 172.21.15.37:0/3502095514 >> 172.21.15.37:0/3502095514 conn(0x5587f1d9e120 msgr2=0x5587f1de67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.086+0000 7f1becbd9740 1 -- 172.21.15.37:0/3502095514 shutdown_connections 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.086+0000 7f1becbd9740 1 -- 172.21.15.37:0/3502095514 wait complete. 2024-04-26T20:24:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.086+0000 7f1becbd9740 20 client.5844 trim_cache size 0 max 16384 2024-04-26T20:24:15.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.104+0000 7f1becbd9740 1 Processor -- start 2024-04-26T20:24:15.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.104+0000 7f1becbd9740 1 -- start start 2024-04-26T20:24:15.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dc8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1de5ce0 0x5587f1df3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1df3fb0 0x5587f1e062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1becbd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1be414beb0 con 0x5587f1dc8440 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1becbd9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41f99f0 con 0x5587f1de5ce0 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1becbd9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1be414b860 con 0x5587f1df3fb0 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1be37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dc8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1be37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dc8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44014/0 (socket says 172.21.15.37:44014) 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1be37fe640 1 -- 172.21.15.37:0/1937227726 learned_addr learned my addr 172.21.15.37:0/1937227726 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1be3fff640 1 --2- 172.21.15.37:0/1937227726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1df3fb0 0x5587f1e062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/1937227726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1de5ce0 0x5587f1df3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1be37fe640 1 -- 172.21.15.37:0/1937227726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1df3fb0 msgr2=0x5587f1e062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1be37fe640 1 --2- 172.21.15.37:0/1937227726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1df3fb0 0x5587f1e062c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1be37fe640 1 -- 172.21.15.37:0/1937227726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1de5ce0 msgr2=0x5587f1df3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1be37fe640 1 --2- 172.21.15.37:0/1937227726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1de5ce0 0x5587f1df3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1be37fe640 1 -- 172.21.15.37:0/1937227726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587f1d943b0 con 0x5587f1dc8440 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1be37fe640 1 --2- 172.21.15.37:0/1937227726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dc8890 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4007870 tx=0x7f1bd4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00be9d29c1ba3d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.105+0000 7f1bc8ff9640 1 -- 172.21.15.37:0/1937227726 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd401b040 con 0x5587f1dc8440 2024-04-26T20:24:15.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.106+0000 7f1bc8ff9640 1 -- 172.21.15.37:0/1937227726 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd401c040 con 0x5587f1dc8440 2024-04-26T20:24:15.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.106+0000 7f1bc8ff9640 1 -- 172.21.15.37:0/1937227726 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4014d30 con 0x5587f1dc8440 2024-04-26T20:24:15.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.106+0000 7f1becbd9740 1 -- 172.21.15.37:0/1937227726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 msgr2=0x5587f1dc8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.106+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1937227726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dc8890 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4007870 tx=0x7f1bd4015000 comp rx=0 tx=0).stop 2024-04-26T20:24:15.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.106+0000 7f1becbd9740 1 -- 172.21.15.37:0/1937227726 shutdown_connections 2024-04-26T20:24:15.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.106+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1937227726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1df3fb0 0x5587f1e062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.106+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1937227726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1de5ce0 0x5587f1df3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.106+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1937227726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dc8890 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.106+0000 7f1becbd9740 1 -- 172.21.15.37:0/1937227726 >> 172.21.15.37:0/1937227726 conn(0x5587f1d9e120 msgr2=0x5587f1dc5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:15.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.106+0000 7f1becbd9740 1 -- 172.21.15.37:0/1937227726 shutdown_connections 2024-04-26T20:24:15.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.106+0000 7f1becbd9740 1 -- 172.21.15.37:0/1937227726 wait complete. 2024-04-26T20:24:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.107+0000 7f1becbd9740 1 -- start start 2024-04-26T20:24:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.107+0000 7f1becbd9740 1 Processor -- start 2024-04-26T20:24:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dfda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1de5ce0 0x5587f1e5daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1becbd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1df3fb0 0x5587f1dc5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1becbd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5587f1df7c40 con 0x5587f1dc8440 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1becbd9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587f1db4ee0 con 0x5587f1df3fb0 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1becbd9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5587f1df7180 con 0x5587f1de5ce0 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1becbd9740 10 client.0 _ll_register_callbacks cb 0x7f1bd80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dfda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dfda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44016/0 (socket says 172.21.15.37:44016) 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be37fe640 1 -- 172.21.15.37:0/1675347124 learned_addr learned my addr 172.21.15.37:0/1675347124 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be3fff640 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1df3fb0 0x5587f1dc5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1de5ce0 0x5587f1e5daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be37fe640 1 -- 172.21.15.37:0/1675347124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1de5ce0 msgr2=0x5587f1e5daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be37fe640 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1de5ce0 0x5587f1e5daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be37fe640 1 -- 172.21.15.37:0/1675347124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1df3fb0 msgr2=0x5587f1dc5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be37fe640 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1df3fb0 0x5587f1dc5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be37fe640 1 -- 172.21.15.37:0/1675347124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587f1de71b0 con 0x5587f1dc8440 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be3fff640 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1df3fb0 0x5587f1dc5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1de5ce0 0x5587f1e5daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1be37fe640 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dfda30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5587f1d98380 tx=0x7f1bd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58a174b83da1be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1bc27fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1bc27fc640 1 -- 172.21.15.37:0/1675347124 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd401b040 con 0x5587f1dc8440 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1bc27fc640 1 -- 172.21.15.37:0/1675347124 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1bd402a2a0 con 0x5587f1dc8440 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.108+0000 7f1becbd9740 1 -- 172.21.15.37:0/1675347124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd401c040 con 0x5587f1dc8440 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1bc27fc640 1 -- 172.21.15.37:0/1675347124 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1bd402a4a0 con 0x5587f1dc8440 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1becbd9740 10 client.5850 fetch_fsmap learned FSMap version 23 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1becbd9740 1 -- 172.21.15.37:0/1675347124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5587f1e5e340 con 0x5587f1dc8440 2024-04-26T20:24:15.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1bc27fc640 1 -- 172.21.15.37:0/1675347124 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd401c040 con 0x5587f1dc8440 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1bc27fc640 1 -- 172.21.15.37:0/1675347124 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f1bd402a640 con 0x5587f1dc8440 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1becbd9740 10 client.5850 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1becbd9740 10 client.5850 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1becbd9740 20 client.5850 populate_metadata read hostname 'smithi037' 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.37:6834/1915220099,v1:172.21.15.37:6835/1915220099] conn(0x5587f1df38a0 0x5587f1e062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1becbd9740 4 client.5850 mds_command: new command op to 5688 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1becbd9740 1 -- 172.21.15.37:0/1675347124 --> [v2:172.21.15.37:6834/1915220099,v1:172.21.15.37:6835/1915220099] -- command(tid 0: {"prefix": "status"}) -- 0x5587f1d0a1c0 con 0x5587f1df38a0 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.37:6834/1915220099,v1:172.21.15.37:6835/1915220099] conn(0x5587f1df38a0 0x5587f1e062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1bc27fc640 10 client.5850 ms_handle_connect on v2:172.21.15.37:6834/1915220099 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1be2ffd640 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.37:6834/1915220099,v1:172.21.15.37:6835/1915220099] conn(0x5587f1df38a0 0x5587f1e062c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919a9bd0346edb05 server_cookie=695dc7cf725ca4ce in_seq=0 out_seq=0 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1bc27fc640 1 -- 172.21.15.37:0/1675347124 <== mds.0 v2:172.21.15.37:6834/1915220099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5587f1d0a1c0 con 0x5587f1df38a0 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1bc27fc640 10 client.5850 handle_command_reply: tid=0 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.109+0000 7f1becbd9740 1 client.5850 shutdown 2024-04-26T20:24:15.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1becbd9740 1 -- 172.21.15.37:0/1675347124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 msgr2=0x5587f1dfda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dfda30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5587f1d98380 tx=0x7f1bd4004730 comp rx=0 tx=0).stop 2024-04-26T20:24:15.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1becbd9740 1 -- 172.21.15.37:0/1675347124 shutdown_connections 2024-04-26T20:24:15.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.37:6834/1915220099,v1:172.21.15.37:6835/1915220099] conn(0x5587f1df38a0 0x5587f1e062c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587f1df3fb0 0x5587f1dc5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5587f1de5ce0 0x5587f1e5daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1becbd9740 1 --2- 172.21.15.37:0/1675347124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587f1dc8440 0x5587f1dfda30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1bc27fc640 0 client.5850 ms_handle_reset on v2:172.21.15.37:6834/1915220099 2024-04-26T20:24:15.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1becbd9740 1 -- 172.21.15.37:0/1675347124 >> 172.21.15.37:0/1675347124 conn(0x5587f1d9e120 msgr2=0x5587f1dc95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:15.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1becbd9740 1 -- 172.21.15.37:0/1675347124 shutdown_connections 2024-04-26T20:24:15.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1becbd9740 1 -- 172.21.15.37:0/1675347124 wait complete. 2024-04-26T20:24:15.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1becbd9740 20 client.5850 trim_cache size 0 max 16384 2024-04-26T20:24:15.118 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:24:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "d49b1c18-e1bf-49f2-acda-0904fa7c6684", 2024-04-26T20:24:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-26T20:24:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5688, 2024-04-26T20:24:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T20:24:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T20:24:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T20:24:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 4.7193994589999999, 2024-04-26T20:24:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 23, 2024-04-26T20:24:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 46, 2024-04-26T20:24:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 46, 2024-04-26T20:24:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 9.7313969290000006 2024-04-26T20:24:15.119 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:24:15.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 -- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1bd006cbc0 msgr2=0x7f1bd006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1bd006cbc0 0x7f1bd006f080 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4006fd0 tx=0x7f1bd4008040 comp rx=0 tx=0).stop 2024-04-26T20:24:15.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 -- 172.21.15.37:0/2355714831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be415f760 msgr2=0x7f1be41f8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be415f760 0x7f1be41f8550 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc007e90 tx=0x7f1bdc00e4b0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 -- 172.21.15.37:0/2355714831 shutdown_connections 2024-04-26T20:24:15.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be415f760 0x7f1be41f8550 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1bd006cbc0 0x7f1bd006f080 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1be415f380 0x7f1be414dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 --2- 172.21.15.37:0/2355714831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be40d5b50 0x7f1be40cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 -- 172.21.15.37:0/2355714831 >> 172.21.15.37:0/2355714831 conn(0x7f1be409e560 msgr2=0x7f1be40b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:15.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 -- 172.21.15.37:0/2355714831 shutdown_connections 2024-04-26T20:24:15.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.110+0000 7f1be9ec4640 1 -- 172.21.15.37:0/2355714831 wait complete. 2024-04-26T20:24:15.122 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:24:15.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.221+0000 7fbbf2645640 1 Processor -- start 2024-04-26T20:24:15.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.221+0000 7fbbf2645640 1 -- start start 2024-04-26T20:24:15.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.222+0000 7fbbf2645640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec073800 0x7fbbec0b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.222+0000 7fbbf2645640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbec070250 con 0x7fbbec0712f0 2024-04-26T20:24:15.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.222+0000 7fbbf2645640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbec070530 con 0x7fbbec144c50 2024-04-26T20:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.222+0000 7fbbf2645640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbbec070810 con 0x7fbbec073800 2024-04-26T20:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.222+0000 7fbbf0be2640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbbec0712f0 0x7fbbec0cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59744/0 (socket says 172.21.15.37:59744) 2024-04-26T20:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.222+0000 7fbbf0be2640 1 -- 172.21.15.37:0/4292438246 learned_addr learned my addr 172.21.15.37:0/4292438246 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeb7fe640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec073800 0x7fbbec0b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422535194 0 0) 0x7fbbec070250 con 0x7fbbec0712f0 2024-04-26T20:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbd4003520 con 0x7fbbec0712f0 2024-04-26T20:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1684609363 0 0) 0x7fbbd4003520 con 0x7fbbec0712f0 2024-04-26T20:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec073800 msgr2=0x7fbbec0b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec073800 0x7fbbec0b6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 >> v1:172.21.15.134:6789/0 conn(0x7fbbec144c50 legacy=0x7fbbec0730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbec070af0 con 0x7fbbec0712f0 2024-04-26T20:24:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fbbe40019f0 con 0x7fbbec0712f0 2024-04-26T20:24:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 >> v1:172.21.15.37:6789/0 conn(0x7fbbec0712f0 legacy=0x7fbbec0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 0x7fbbd4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec073800 0x7fbbd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd4004d40 0x7fbbd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec070810 con 0x7fbbd40039e0 2024-04-26T20:24:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd400dc00 con 0x7fbbd4004d40 2024-04-26T20:24:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd400de00 con 0x7fbbec073800 2024-04-26T20:24:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbebfff640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 0x7fbbd4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbeb7fe640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec073800 0x7fbbd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.223+0000 7fbbf0be2640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd4004d40 0x7fbbd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.224+0000 7fbbebfff640 1 -- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec073800 msgr2=0x7fbbd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.224+0000 7fbbebfff640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec073800 0x7fbbd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.224+0000 7fbbebfff640 1 -- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd4004d40 msgr2=0x7fbbd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.224+0000 7fbbebfff640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd4004d40 0x7fbbd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.224+0000 7fbbebfff640 1 -- 172.21.15.37:0/4292438246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd4003520 con 0x7fbbd40039e0 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.224+0000 7fbbf0be2640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd4004d40 0x7fbbd40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.224+0000 7fbbebfff640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 0x7fbbd4003dc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc001860 tx=0x7fbbd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1a3bb8ac287126 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.224+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc037070 con 0x7fbbd40039e0 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.224+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbdc02ab60 con 0x7fbbd40039e0 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.224+0000 7fbbeaffd640 1 -- 172.21.15.37:0/4292438246 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc001430 con 0x7fbbd40039e0 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.225+0000 7fbbf2645640 1 -- 172.21.15.37:0/4292438246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 msgr2=0x7fbbd4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.225+0000 7fbbf2645640 1 --2- 172.21.15.37:0/4292438246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 0x7fbbd4003dc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc001860 tx=0x7fbbd4005830 comp rx=0 tx=0).stop 2024-04-26T20:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.225+0000 7fbbf0be2640 1 -- 172.21.15.37:0/4292438246 reap_dead start 2024-04-26T20:24:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.225+0000 7fbbf2645640 1 -- 172.21.15.37:0/4292438246 shutdown_connections 2024-04-26T20:24:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.225+0000 7fbbf2645640 1 -- 172.21.15.37:0/4292438246 >> 172.21.15.37:0/4292438246 conn(0x7fbbec0b3b00 msgr2=0x7fbbec0b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.225+0000 7fbbf2645640 1 -- 172.21.15.37:0/4292438246 shutdown_connections 2024-04-26T20:24:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.225+0000 7fbbf2645640 1 -- 172.21.15.37:0/4292438246 wait complete. 2024-04-26T20:24:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.226+0000 7fbbf2645640 1 Processor -- start 2024-04-26T20:24:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.226+0000 7fbbf2645640 1 -- start start 2024-04-26T20:24:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbf2645640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 0x7fbbec15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbf2645640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbec15b8a0 0x7fbbec137200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbf2645640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec138a80 0x7fbbec13ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbf2645640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec145030 con 0x7fbbd40039e0 2024-04-26T20:24:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbf2645640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec145b00 con 0x7fbbec15b8a0 2024-04-26T20:24:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbf2645640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbbec1451b0 con 0x7fbbec138a80 2024-04-26T20:24:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 0x7fbbec15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 0x7fbbec15eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44034/0 (socket says 172.21.15.37:44034) 2024-04-26T20:24:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbebfff640 1 -- 172.21.15.37:0/1774402868 learned_addr learned my addr 172.21.15.37:0/1774402868 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbeb7fe640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbec15b8a0 0x7fbbec137200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbf0be2640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec138a80 0x7fbbec13ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbebfff640 1 -- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec138a80 msgr2=0x7fbbec13ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.227+0000 7fbbebfff640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec138a80 0x7fbbec13ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.228+0000 7fbbebfff640 1 -- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbec15b8a0 msgr2=0x7fbbec137200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.228+0000 7fbbebfff640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbec15b8a0 0x7fbbec137200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.228+0000 7fbbebfff640 1 -- 172.21.15.37:0/1774402868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbec137740 con 0x7fbbd40039e0 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.228+0000 7fbbeb7fe640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbec15b8a0 0x7fbbec137200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.228+0000 7fbbebfff640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 0x7fbbec15eaf0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc02a5f0 tx=0x7fbbdc001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.228+0000 7fbbe8ff9640 1 -- 172.21.15.37:0/1774402868 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc03f020 con 0x7fbbd40039e0 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.228+0000 7fbbe8ff9640 1 -- 172.21.15.37:0/1774402868 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbdc005270 con 0x7fbbd40039e0 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.228+0000 7fbbf2645640 1 -- 172.21.15.37:0/1774402868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbec15bd20 con 0x7fbbd40039e0 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.228+0000 7fbbe8ff9640 1 -- 172.21.15.37:0/1774402868 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc037040 con 0x7fbbd40039e0 2024-04-26T20:24:15.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.228+0000 7fbbf2645640 1 -- 172.21.15.37:0/1774402868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbec137cc0 con 0x7fbbd40039e0 2024-04-26T20:24:15.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.229+0000 7fbbf2645640 1 -- 172.21.15.37:0/1774402868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbec0693e0 con 0x7fbbd40039e0 2024-04-26T20:24:15.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.229+0000 7fbbe8ff9640 1 -- 172.21.15.37:0/1774402868 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fbbdc042020 con 0x7fbbd40039e0 2024-04-26T20:24:15.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.230+0000 7fbbe8ff9640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbbd806cbc0 0x7fbbd806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.230+0000 7fbbe8ff9640 1 -- 172.21.15.37:0/1774402868 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fbbdc0ac000 con 0x7fbbd40039e0 2024-04-26T20:24:15.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.230+0000 7fbbeb7fe640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbbd806cbc0 0x7fbbd806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.230+0000 7fbbeb7fe640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbbd806cbc0 0x7fbbd806f080 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbbec15bfe0 tx=0x7fbbe0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:15.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.234+0000 7fbbe8ff9640 1 -- 172.21.15.37:0/1774402868 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbbdc049030 con 0x7fbbd40039e0 2024-04-26T20:24:15.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.457+0000 7fbbf2645640 1 -- 172.21.15.37:0/1774402868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbbec0d2670 con 0x7fbbd40039e0 2024-04-26T20:24:15.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.458+0000 7fbbe8ff9640 1 -- 172.21.15.37:0/1774402868 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7fbbdc079e40 con 0x7fbbd40039e0 2024-04-26T20:24:15.459 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:24:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 -- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbbd806cbc0 msgr2=0x7fbbd806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbbd806cbc0 0x7fbbd806f080 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbbec15bfe0 tx=0x7fbbe0008040 comp rx=0 tx=0).stop 2024-04-26T20:24:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 -- 172.21.15.37:0/1774402868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 msgr2=0x7fbbec15eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 0x7fbbec15eaf0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc02a5f0 tx=0x7fbbdc001230 comp rx=0 tx=0).stop 2024-04-26T20:24:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 -- 172.21.15.37:0/1774402868 shutdown_connections 2024-04-26T20:24:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbbd806cbc0 0x7fbbd806f080 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbec138a80 0x7fbbec13ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbec15b8a0 0x7fbbec137200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 --2- 172.21.15.37:0/1774402868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbd40039e0 0x7fbbec15eaf0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:15.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 -- 172.21.15.37:0/1774402868 >> 172.21.15.37:0/1774402868 conn(0x7fbbec0b3b00 msgr2=0x7fbbec0b6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:15.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 -- 172.21.15.37:0/1774402868 shutdown_connections 2024-04-26T20:24:15.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15.460+0000 7fbbf2645640 1 -- 172.21.15.37:0/1774402868 wait complete. 2024-04-26T20:24:15.472 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T20:24:15.472 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:24:15.472 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-26T20:24:15.472 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:24:15.472 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:24:15.472 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T20:24:15.472 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T20:24:15.472 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:24:15.472 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:24:15.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:15.472 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-26T20:24:15.488 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:24:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:24:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:24:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-26T20:24:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:15.488 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:24:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42589sec preferred_lft 42589sec 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:24:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 06:a8:3c:e3:8e:a4 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::247e:2bff:fe9c:6859/64 scope link 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 06:a8:3c:e3:8e:a4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::4a8:3cff:fee3:8ea4/64 scope link 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 96:92:77:52:91:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:24:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::9492:77ff:fe52:9118/64 scope link 2024-04-26T20:24:15.491 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:15.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:15.491 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:24:15.552 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-26T20:24:15.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:15.554 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-26T20:24:15.613 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-26T20:24:15.613 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:24:15.613 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:24:15.613 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:24:15.613 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:24:15.614 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:24:15.614 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T20:24:15.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:24:15.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:24:15.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:24:15.614 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T20:24:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:24:15.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:24:15.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:15.740 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T20:24:15.755 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:24:15.755 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:24:15.756 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:24:15.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:15.756 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-26T20:24:15.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:24:15.820 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:24:15.876 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:24:15.877 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:24:15.877 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:24:15.931 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T20:24:15.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:15.932 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:24:16.014 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:24:16.015 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:24:16.022 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:24:16.022 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:24:16.022 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-04-26T20:24:16.022 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with new device syntax: 0@d49b1c18-e1bf-49f2-acda-0904fa7c6684.cephfs=/ 2024-04-26T20:24:16.022 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.37:3300/172.21.15.134:3300/172.21.15.134:3301" will pass to kernel 2024-04-26T20:24:16.023 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T20:24:16.023 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:24:16.023 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:24:16.023 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:24:16.023 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:24:16.025 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:24:16.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:16.025 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:24:16.090 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:24:16.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:24:16.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:24:16.191 INFO:teuthology.orchestra.run.smithi037.stdout:client5138 2024-04-26T20:24:16.192 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:24:16.192 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:24:16.208 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:24:16.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.347+0000 7f0cf50dc640 1 Processor -- start 2024-04-26T20:24:16.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.347+0000 7f0cf50dc640 1 -- start start 2024-04-26T20:24:16.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.348+0000 7f0cf50dc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0cf0162b40 0x7f0cf00d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:16.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.348+0000 7f0cf50dc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00d3c80 0x7f0cf00d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:16.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.348+0000 7f0cf50dc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf00d0210 con 0x7f0cf00d3820 2024-04-26T20:24:16.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.348+0000 7f0cf50dc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf00d04f0 con 0x7f0cf0162b40 2024-04-26T20:24:16.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.348+0000 7f0cf50dc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf00d07d0 con 0x7f0cf00d3c80 2024-04-26T20:24:16.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.348+0000 7f0ceed9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0cf0162b40 0x7f0cf00d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:16.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.348+0000 7f0cee59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0cf00d3820 0x7f0cf00d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47078/0 (socket says 172.21.15.37:47078) 2024-04-26T20:24:16.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.348+0000 7f0cee59c640 1 -- 172.21.15.37:0/853945232 learned_addr learned my addr 172.21.15.37:0/853945232 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:16.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.349+0000 7f0cef59e640 1 --2- 172.21.15.37:0/853945232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00d3c80 0x7f0cf00d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.349+0000 7f0ceed9d640 1 -- 172.21.15.37:0/853945232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00d3c80 msgr2=0x7f0cf00d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.349+0000 7f0ceed9d640 1 --2- 172.21.15.37:0/853945232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00d3c80 0x7f0cf00d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.349+0000 7f0ceed9d640 1 -- 172.21.15.37:0/853945232 >> v1:172.21.15.37:6789/0 conn(0x7f0cf00d3820 legacy=0x7f0cf00d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.349+0000 7f0ceed9d640 1 -- 172.21.15.37:0/853945232 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf00d0ab0 con 0x7f0cf0162b40 2024-04-26T20:24:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.349+0000 7f0ceed9d640 1 --2- 172.21.15.37:0/853945232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf0162b40 0x7f0cf00d5bd0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80097a0 tx=0x7f0ce802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1705fd6eb86b685b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.349+0000 7f0cedd9b640 1 -- 172.21.15.37:0/853945232 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8037070 con 0x7f0cf0162b40 2024-04-26T20:24:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.350+0000 7f0cedd9b640 1 -- 172.21.15.37:0/853945232 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce8032440 con 0x7f0cf0162b40 2024-04-26T20:24:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.350+0000 7f0cedd9b640 1 -- 172.21.15.37:0/853945232 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8032810 con 0x7f0cf0162b40 2024-04-26T20:24:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.350+0000 7f0cf50dc640 1 -- 172.21.15.37:0/853945232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf0162b40 msgr2=0x7f0cf00d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.350+0000 7f0cf50dc640 1 --2- 172.21.15.37:0/853945232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf0162b40 0x7f0cf00d5bd0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80097a0 tx=0x7f0ce802a210 comp rx=0 tx=0).stop 2024-04-26T20:24:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.350+0000 7f0cf50dc640 1 -- 172.21.15.37:0/853945232 shutdown_connections 2024-04-26T20:24:16.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.350+0000 7f0cf50dc640 1 --2- 172.21.15.37:0/853945232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00d3c80 0x7f0cf00d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.350+0000 7f0cf50dc640 1 --2- 172.21.15.37:0/853945232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf0162b40 0x7f0cf00d5bd0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.350+0000 7f0cf50dc640 1 -- 172.21.15.37:0/853945232 >> 172.21.15.37:0/853945232 conn(0x7f0cf009e550 msgr2=0x7f0cf00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:16.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.350+0000 7f0cf50dc640 1 -- 172.21.15.37:0/853945232 shutdown_connections 2024-04-26T20:24:16.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.350+0000 7f0cf50dc640 1 -- 172.21.15.37:0/853945232 wait complete. 2024-04-26T20:24:16.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.352+0000 7f0cf50dc640 1 Processor -- start 2024-04-26T20:24:16.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.352+0000 7f0cf50dc640 1 -- start start 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.352+0000 7f0cf50dc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf00d3820 0x7f0cf00ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.352+0000 7f0cf50dc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00d3c80 0x7f0cf015ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.352+0000 7f0cf50dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf0162b40 0x7f0cf015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.352+0000 7f0cf50dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf00b5bb0 con 0x7f0cf0162b40 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.352+0000 7f0cf50dc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf00d4110 con 0x7f0cf00d3820 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.352+0000 7f0cf50dc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf00d6110 con 0x7f0cf00d3c80 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.352+0000 7f0cef59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf0162b40 0x7f0cf015f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cef59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf0162b40 0x7f0cf015f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52846/0 (socket says 172.21.15.37:52846) 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cef59e640 1 -- 172.21.15.37:0/2924495848 learned_addr learned my addr 172.21.15.37:0/2924495848 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0ceed9d640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf00d3820 0x7f0cf00ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cee59c640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00d3c80 0x7f0cf015ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cef59e640 1 -- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00d3c80 msgr2=0x7f0cf015ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:16.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cef59e640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00d3c80 0x7f0cf015ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cef59e640 1 -- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf00d3820 msgr2=0x7f0cf00ba6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cef59e640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf00d3820 0x7f0cf00ba6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cef59e640 1 -- 172.21.15.37:0/2924495848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf015fb70 con 0x7f0cf0162b40 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0ceed9d640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf00d3820 0x7f0cf00ba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cee59c640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00d3c80 0x7f0cf015ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cef59e640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf0162b40 0x7f0cf015f4d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0007e60 tx=0x7f0ce000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0ce77fe640 1 -- 172.21.15.37:0/2924495848 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0014420 con 0x7f0cf0162b40 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0ce77fe640 1 -- 172.21.15.37:0/2924495848 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce0019070 con 0x7f0cf0162b40 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0ce77fe640 1 -- 172.21.15.37:0/2924495848 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0014820 con 0x7f0cf0162b40 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cf50dc640 1 -- 172.21.15.37:0/2924495848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf015fe50 con 0x7f0cf0162b40 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.353+0000 7f0cf50dc640 1 -- 172.21.15.37:0/2924495848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf0160410 con 0x7f0cf0162b40 2024-04-26T20:24:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.354+0000 7f0cf50dc640 1 -- 172.21.15.37:0/2924495848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf00d2750 con 0x7f0cf0162b40 2024-04-26T20:24:16.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.355+0000 7f0ce77fe640 1 -- 172.21.15.37:0/2924495848 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f0ce00149c0 con 0x7f0cf0162b40 2024-04-26T20:24:16.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.355+0000 7f0ce77fe640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0cd006cbc0 0x7f0cd006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:16.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.355+0000 7f0ce77fe640 1 -- 172.21.15.37:0/2924495848 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f0ce008c730 con 0x7f0cf0162b40 2024-04-26T20:24:16.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.355+0000 7f0ceed9d640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0cd006cbc0 0x7f0cd006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:16.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.356+0000 7f0ceed9d640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0cd006cbc0 0x7f0cd006f080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8034000 tx=0x7f0ce802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:16.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.359+0000 7f0ce77fe640 1 -- 172.21.15.37:0/2924495848 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0cf00d2750 con 0x7f0cf0162b40 2024-04-26T20:24:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.588+0000 7f0cf50dc640 1 -- 172.21.15.37:0/2924495848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0cf00d47d0 con 0x7f0cf0162b40 2024-04-26T20:24:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.588+0000 7f0ce77fe640 1 -- 172.21.15.37:0/2924495848 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f0cf00d47d0 con 0x7f0cf0162b40 2024-04-26T20:24:16.590 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:24:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.590+0000 7f0cf50dc640 1 -- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0cd006cbc0 msgr2=0x7f0cd006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.591+0000 7f0cf50dc640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0cd006cbc0 0x7f0cd006f080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8034000 tx=0x7f0ce802e000 comp rx=0 tx=0).stop 2024-04-26T20:24:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.591+0000 7f0cf50dc640 1 -- 172.21.15.37:0/2924495848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf0162b40 msgr2=0x7f0cf015f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.591+0000 7f0cf50dc640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf0162b40 0x7f0cf015f4d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0007e60 tx=0x7f0ce000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.591+0000 7f0cf50dc640 1 -- 172.21.15.37:0/2924495848 shutdown_connections 2024-04-26T20:24:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.591+0000 7f0cf50dc640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf0162b40 0x7f0cf015f4d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.591+0000 7f0cf50dc640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f0cd006cbc0 0x7f0cd006f080 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.591+0000 7f0cf50dc640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00d3c80 0x7f0cf015ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.591+0000 7f0cf50dc640 1 --2- 172.21.15.37:0/2924495848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf00d3820 0x7f0cf00ba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.591+0000 7f0cf50dc640 1 -- 172.21.15.37:0/2924495848 >> 172.21.15.37:0/2924495848 conn(0x7f0cf009e550 msgr2=0x7f0cf00b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:16.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.591+0000 7f0cf50dc640 1 -- 172.21.15.37:0/2924495848 shutdown_connections 2024-04-26T20:24:16.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:16.591+0000 7f0cf50dc640 1 -- 172.21.15.37:0/2924495848 wait complete. 2024-04-26T20:24:16.602 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T20:24:16.603 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:24:16.603 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi134.front.sepia.ceph.com 2024-04-26T20:24:16.603 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T20:24:16.603 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:24:16.603 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T20:24:16.603 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T20:24:16.603 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:24:16.603 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:24:16.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:16.603 DEBUG:teuthology.orchestra.run.smithi134:> ip addr 2024-04-26T20:24:16.622 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:24:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:24:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e6 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f0 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e7 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f1 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e8 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:24:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e9 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.21.15.134/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft 42617sec preferred_lft 42617sec 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::5526:381c:36db:a6d8/64 scope link noprefixroute 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0a:82:83:43:1c:3f brd ff:ff:ff:ff:ff:ff 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::205c:9aff:fee0:b7b/64 scope link 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether c2:13:60:38:f8:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::c013:60ff:fe38:f842/64 scope link 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:16.624 INFO:teuthology.orchestra.run.smithi134.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:24:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0a:82:83:43:1c:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:24:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::882:83ff:fe43:1c3f/64 scope link 2024-04-26T20:24:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:24:16.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:16.625 DEBUG:teuthology.orchestra.run.smithi134:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:24:16.689 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-04-26T20:24:16.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:16.691 DEBUG:teuthology.orchestra.run.smithi134:> route 2024-04-26T20:24:16.749 INFO:teuthology.orchestra.run.smithi134.stdout:Kernel IP routing table 2024-04-26T20:24:16.749 INFO:teuthology.orchestra.run.smithi134.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:24:16.749 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:24:16.749 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:24:16.749 INFO:teuthology.orchestra.run.smithi134.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:24:16.750 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:24:16.750 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-26T20:24:16.750 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:24:16.750 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:24:16.750 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:24:16.750 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-26T20:24:16.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:24:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:24:16.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:24:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:24:16.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:16.880 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-26T20:24:16.895 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T20:24:16.895 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:24:16.895 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:24:16.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:16.895 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set brx.0 up 2024-04-26T20:24:16.961 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:24:16.961 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T20:24:17.017 INFO:teuthology.orchestra.run.smithi134.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:24:17.018 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:24:17.018 DEBUG:teuthology.orchestra.run.smithi134:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:24:17.071 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-26T20:24:17.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:17.072 DEBUG:teuthology.orchestra.run.smithi134:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:24:17.151 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:24:17.151 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:24:17.159 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T20:24:17.159 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T20:24:17.159 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-04-26T20:24:17.159 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: trying mount with new device syntax: 1@d49b1c18-e1bf-49f2-acda-0904fa7c6684.cephfs=/ 2024-04-26T20:24:17.159 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.37:3300/172.21.15.134:3300/172.21.15.134:3301" will pass to kernel 2024-04-26T20:24:17.160 INFO:teuthology.orchestra.run.smithi134.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T20:24:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:24:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:24:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:24:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:24:17.161 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:24:17.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:24:17.161 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:24:17.225 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:24:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:24:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:24:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-26T20:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:24:17.319 INFO:teuthology.orchestra.run.smithi134.stdout:client5458 2024-04-26T20:24:17.319 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:24:17.319 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:24:17.334 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-26T20:24:17.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.452+0000 7fc96dcda640 1 Processor -- start 2024-04-26T20:24:17.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.453+0000 7fc96dcda640 1 -- start start 2024-04-26T20:24:17.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.453+0000 7fc96dcda640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc96815f2b0 0x7fc9680cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:17.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.453+0000 7fc96dcda640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9680b1e50 0x7fc9680b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:17.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.453+0000 7fc96dcda640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc96806e9c0 0x7fc96806ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:17.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.453+0000 7fc96dcda640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc9680d27a0 con 0x7fc96815f2b0 2024-04-26T20:24:17.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.453+0000 7fc96dcda640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc9680d2a60 con 0x7fc96806e9c0 2024-04-26T20:24:17.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.453+0000 7fc96dcda640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc9680d2d40 con 0x7fc9680b1e50 2024-04-26T20:24:17.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.453+0000 7fc9677fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc96815f2b0 0x7fc9680cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:17.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.454+0000 7fc9677fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc96815f2b0 0x7fc9680cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52852/0 (socket says 172.21.15.37:52852) 2024-04-26T20:24:17.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.454+0000 7fc9677fe640 1 -- 172.21.15.37:0/1684809502 learned_addr learned my addr 172.21.15.37:0/1684809502 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:17.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.454+0000 7fc967fff640 1 --2- 172.21.15.37:0/1684809502 >> v2:172.21.15.134:3300/0 conn(0x7fc96806e9c0 0x7fc96806ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:17.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.454+0000 7fc966ffd640 1 --2- 172.21.15.37:0/1684809502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9680b1e50 0x7fc9680b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:17.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.454+0000 7fc9677fe640 1 -- 172.21.15.37:0/1684809502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9680b1e50 msgr2=0x7fc9680b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:17.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.454+0000 7fc9677fe640 1 --2- 172.21.15.37:0/1684809502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9680b1e50 0x7fc9680b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:17.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.454+0000 7fc9677fe640 1 -- 172.21.15.37:0/1684809502 >> v2:172.21.15.134:3300/0 conn(0x7fc96806e9c0 msgr2=0x7fc96806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:17.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.454+0000 7fc9677fe640 1 --2- 172.21.15.37:0/1684809502 >> v2:172.21.15.134:3300/0 conn(0x7fc96806e9c0 0x7fc96806ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:17.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.454+0000 7fc9677fe640 1 -- 172.21.15.37:0/1684809502 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9680d0bf0 con 0x7fc96815f2b0 2024-04-26T20:24:17.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.454+0000 7fc966ffd640 1 --2- 172.21.15.37:0/1684809502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9680b1e50 0x7fc9680b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:17.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc9677fe640 1 --2- 172.21.15.37:0/1684809502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc96815f2b0 0x7fc9680cd5d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc96806e780 tx=0x7fc95802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d541c4904f4798 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:17.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc9667fc640 1 -- 172.21.15.37:0/1684809502 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958037070 con 0x7fc96815f2b0 2024-04-26T20:24:17.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc9667fc640 1 -- 172.21.15.37:0/1684809502 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc958032440 con 0x7fc96815f2b0 2024-04-26T20:24:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc9667fc640 1 -- 172.21.15.37:0/1684809502 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958032810 con 0x7fc96815f2b0 2024-04-26T20:24:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc96dcda640 1 -- 172.21.15.37:0/1684809502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc96815f2b0 msgr2=0x7fc9680cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc96dcda640 1 --2- 172.21.15.37:0/1684809502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc96815f2b0 0x7fc9680cd5d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc96806e780 tx=0x7fc95802a140 comp rx=0 tx=0).stop 2024-04-26T20:24:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc96dcda640 1 -- 172.21.15.37:0/1684809502 shutdown_connections 2024-04-26T20:24:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc96dcda640 1 --2- 172.21.15.37:0/1684809502 >> v2:172.21.15.134:3300/0 conn(0x7fc96806e9c0 0x7fc96806ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc96dcda640 1 --2- 172.21.15.37:0/1684809502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9680b1e50 0x7fc9680b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc96dcda640 1 --2- 172.21.15.37:0/1684809502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc96815f2b0 0x7fc9680cd5d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc96dcda640 1 -- 172.21.15.37:0/1684809502 >> 172.21.15.37:0/1684809502 conn(0x7fc96806c870 msgr2=0x7fc968068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc96dcda640 1 -- 172.21.15.37:0/1684809502 shutdown_connections 2024-04-26T20:24:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.455+0000 7fc96dcda640 1 -- 172.21.15.37:0/1684809502 wait complete. 2024-04-26T20:24:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.457+0000 7fc96dcda640 1 Processor -- start 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.457+0000 7fc96dcda640 1 -- start start 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc96dcda640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc96806e9c0 0x7fc9681f9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc96dcda640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9680b1e50 0x7fc9681f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc96dcda640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc96815f2b0 0x7fc9680caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc96dcda640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc96815f690 con 0x7fc9680b1e50 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc96dcda640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9680d1240 con 0x7fc96806e9c0 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc96dcda640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc96815f810 con 0x7fc96815f2b0 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc966ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9680b1e50 0x7fc9681f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc966ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9680b1e50 0x7fc9681f7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52862/0 (socket says 172.21.15.37:52862) 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc966ffd640 1 -- 172.21.15.37:0/706838912 learned_addr learned my addr 172.21.15.37:0/706838912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc9677fe640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc96806e9c0 0x7fc9681f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:17.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc967fff640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc96815f2b0 0x7fc9680caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc966ffd640 1 -- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc96815f2b0 msgr2=0x7fc9680caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc966ffd640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc96815f2b0 0x7fc9680caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc966ffd640 1 -- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc96806e9c0 msgr2=0x7fc9681f9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc966ffd640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc96806e9c0 0x7fc9681f9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc966ffd640 1 -- 172.21.15.37:0/706838912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9681f84f0 con 0x7fc9680b1e50 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc966ffd640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9680b1e50 0x7fc9681f7fb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00d7d0 tx=0x7fc95c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.458+0000 7fc94ffff640 1 -- 172.21.15.37:0/706838912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c004030 con 0x7fc9680b1e50 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.459+0000 7fc94ffff640 1 -- 172.21.15.37:0/706838912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c0041d0 con 0x7fc9680b1e50 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.459+0000 7fc96dcda640 1 -- 172.21.15.37:0/706838912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9680cb290 con 0x7fc9680b1e50 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.459+0000 7fc94ffff640 1 -- 172.21.15.37:0/706838912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00b6c0 con 0x7fc9680b1e50 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.459+0000 7fc96dcda640 1 -- 172.21.15.37:0/706838912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96815fd10 con 0x7fc9680b1e50 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.460+0000 7fc96dcda640 1 -- 172.21.15.37:0/706838912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9680693e0 con 0x7fc9680b1e50 2024-04-26T20:24:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.460+0000 7fc94ffff640 1 -- 172.21.15.37:0/706838912 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fc95c00b860 con 0x7fc9680b1e50 2024-04-26T20:24:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.460+0000 7fc94ffff640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc95406cbf0 0x7fc95406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.461+0000 7fc94ffff640 1 -- 172.21.15.37:0/706838912 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fc95c08c0b0 con 0x7fc9680b1e50 2024-04-26T20:24:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.461+0000 7fc9677fe640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc95406cbf0 0x7fc95406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.461+0000 7fc9677fe640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc95406cbf0 0x7fc95406f0b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc958034000 tx=0x7fc95802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.465+0000 7fc94ffff640 1 -- 172.21.15.37:0/706838912 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc9681f84f0 con 0x7fc9680b1e50 2024-04-26T20:24:17.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:17.693+0000 7fc96dcda640 1 -- 172.21.15.37:0/706838912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fc968068230 con 0x7fc9680b1e50 2024-04-26T20:24:18.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.441+0000 7fc94ffff640 1 -- 172.21.15.37:0/706838912 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v24) ==== 136+0+0 (secure 0 0 0) 0x7fc95c059b40 con 0x7fc9680b1e50 2024-04-26T20:24:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 -- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc95406cbf0 msgr2=0x7fc95406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc95406cbf0 0x7fc95406f0b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc958034000 tx=0x7fc95802e000 comp rx=0 tx=0).stop 2024-04-26T20:24:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 -- 172.21.15.37:0/706838912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9680b1e50 msgr2=0x7fc9681f7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9680b1e50 0x7fc9681f7fb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00d7d0 tx=0x7fc95c00dc90 comp rx=0 tx=0).stop 2024-04-26T20:24:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 -- 172.21.15.37:0/706838912 shutdown_connections 2024-04-26T20:24:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc95406cbf0 0x7fc95406f0b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc96815f2b0 0x7fc9680caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9680b1e50 0x7fc9681f7fb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 --2- 172.21.15.37:0/706838912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc96806e9c0 0x7fc9681f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 -- 172.21.15.37:0/706838912 >> 172.21.15.37:0/706838912 conn(0x7fc96806c870 msgr2=0x7fc9680cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 -- 172.21.15.37:0/706838912 shutdown_connections 2024-04-26T20:24:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.444+0000 7fc96dcda640 1 -- 172.21.15.37:0/706838912 wait complete. 2024-04-26T20:24:18.458 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-26T20:24:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.565+0000 7f93d98f8640 1 Processor -- start 2024-04-26T20:24:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.565+0000 7f93d98f8640 1 -- start start 2024-04-26T20:24:18.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.566+0000 7f93d98f8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 0x7f93d40b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.566+0000 7f93d98f8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93d415f760 0x7f93d40b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.566+0000 7f93d98f8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d40d3de0 con 0x7f93d415f380 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.566+0000 7f93d98f8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f93d40d43a0 con 0x7f93d415f760 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.566+0000 7f93d98f8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93d40d40c0 con 0x7f93d40d22e0 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d37fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93d415f760 0x7f93d40b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d27fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 0x7f93d40b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d2ffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f93d415f380 0x7f93d40d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:42016/0 (socket says 172.21.15.37:42016) 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d2ffd640 1 -- 172.21.15.37:0/3180641302 learned_addr learned my addr 172.21.15.37:0/3180641302 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d37fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93d415f760 0x7f93d40b8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:43332/0 (socket says 172.21.15.37:43332) 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d27fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 0x7f93d40b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:46332/0 (socket says 172.21.15.37:46332) 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d1ffb640 1 -- 172.21.15.37:0/3180641302 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 132816948 0 0) 0x7f93d40d3de0 con 0x7f93d415f380 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d1ffb640 1 -- 172.21.15.37:0/3180641302 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93c0003520 con 0x7f93d415f380 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d27fc640 1 -- 172.21.15.37:0/3180641302 >> v2:172.21.15.134:3300/0 conn(0x7f93d415f760 msgr2=0x7f93d40b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d27fc640 1 --2- 172.21.15.37:0/3180641302 >> v2:172.21.15.134:3300/0 conn(0x7f93d415f760 0x7f93d40b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d27fc640 1 -- 172.21.15.37:0/3180641302 >> v1:172.21.15.134:6789/0 conn(0x7f93d415f380 legacy=0x7f93d40d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d27fc640 1 -- 172.21.15.37:0/3180641302 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d40b8280 con 0x7f93d40d22e0 2024-04-26T20:24:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.567+0000 7f93d37fe640 1 --2- 172.21.15.37:0/3180641302 >> v2:172.21.15.134:3300/0 conn(0x7f93d415f760 0x7f93d40b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.568+0000 7f93d27fc640 1 --2- 172.21.15.37:0/3180641302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 0x7f93d40b8a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0097a0 tx=0x7f93bc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3573fd3fd1632a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.568+0000 7f93d1ffb640 1 -- 172.21.15.37:0/3180641302 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc037070 con 0x7f93d40d22e0 2024-04-26T20:24:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.568+0000 7f93d1ffb640 1 -- 172.21.15.37:0/3180641302 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93bc032440 con 0x7f93d40d22e0 2024-04-26T20:24:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.568+0000 7f93d1ffb640 1 -- 172.21.15.37:0/3180641302 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0327f0 con 0x7f93d40d22e0 2024-04-26T20:24:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.569+0000 7f93d98f8640 1 -- 172.21.15.37:0/3180641302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 msgr2=0x7f93d40b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.569+0000 7f93d98f8640 1 --2- 172.21.15.37:0/3180641302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 0x7f93d40b8a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0097a0 tx=0x7f93bc02a210 comp rx=0 tx=0).stop 2024-04-26T20:24:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.569+0000 7f93d98f8640 1 -- 172.21.15.37:0/3180641302 shutdown_connections 2024-04-26T20:24:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.569+0000 7f93d98f8640 1 --2- 172.21.15.37:0/3180641302 >> v2:172.21.15.134:3300/0 conn(0x7f93d415f760 0x7f93d40b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.569+0000 7f93d98f8640 1 --2- 172.21.15.37:0/3180641302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 0x7f93d40b8a80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.569+0000 7f93d98f8640 1 -- 172.21.15.37:0/3180641302 >> 172.21.15.37:0/3180641302 conn(0x7f93d409e560 msgr2=0x7f93d40d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.569+0000 7f93d98f8640 1 -- 172.21.15.37:0/3180641302 shutdown_connections 2024-04-26T20:24:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.569+0000 7f93d98f8640 1 -- 172.21.15.37:0/3180641302 wait complete. 2024-04-26T20:24:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.570+0000 7f93d98f8640 1 Processor -- start 2024-04-26T20:24:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.570+0000 7f93d98f8640 1 -- start start 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d98f8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 0x7f93d41627f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d98f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d415f380 0x7f93d414b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d98f8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93d415f760 0x7f93d40c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d98f8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40d1280 con 0x7f93d415f380 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d98f8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40d1100 con 0x7f93d415f760 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d98f8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93d40d5b50 con 0x7f93d40d22e0 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d415f380 0x7f93d414b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d415f380 0x7f93d414b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52876/0 (socket says 172.21.15.37:52876) 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d27fc640 1 -- 172.21.15.37:0/1540338730 learned_addr learned my addr 172.21.15.37:0/1540338730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d2ffd640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 0x7f93d41627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d37fe640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93d415f760 0x7f93d40c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d27fc640 1 -- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 msgr2=0x7f93d41627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d27fc640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 0x7f93d41627f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d27fc640 1 -- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93d415f760 msgr2=0x7f93d40c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d27fc640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93d415f760 0x7f93d40c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d27fc640 1 -- 172.21.15.37:0/1540338730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d415ff70 con 0x7f93d415f380 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.571+0000 7f93d37fe640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93d415f760 0x7f93d40c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.572+0000 7f93d27fc640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d415f380 0x7f93d414b240 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f93bc02a250 tx=0x7f93bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.572+0000 7f93cbfff640 1 -- 172.21.15.37:0/1540338730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc037040 con 0x7f93d415f380 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.572+0000 7f93cbfff640 1 -- 172.21.15.37:0/1540338730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93bc042070 con 0x7f93d415f380 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.572+0000 7f93d98f8640 1 -- 172.21.15.37:0/1540338730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d4160210 con 0x7f93d415f380 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.572+0000 7f93cbfff640 1 -- 172.21.15.37:0/1540338730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc03b930 con 0x7f93d415f380 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.572+0000 7f93d98f8640 1 -- 172.21.15.37:0/1540338730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d40ca280 con 0x7f93d415f380 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.573+0000 7f93d98f8640 1 -- 172.21.15.37:0/1540338730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d406a360 con 0x7f93d415f380 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.573+0000 7f93cbfff640 1 -- 172.21.15.37:0/1540338730 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f93bc03bad0 con 0x7f93d415f380 2024-04-26T20:24:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.574+0000 7f93cbfff640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f93b406cbc0 0x7f93b406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.574+0000 7f93cbfff640 1 -- 172.21.15.37:0/1540338730 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f93bc0ac890 con 0x7f93d415f380 2024-04-26T20:24:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.574+0000 7f93d2ffd640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f93b406cbc0 0x7f93b406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.574+0000 7f93d2ffd640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f93b406cbc0 0x7f93b406f080 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f93c4005c00 tx=0x7f93c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.578+0000 7f93cbfff640 1 -- 172.21.15.37:0/1540338730 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93bc07a970 con 0x7f93d415f380 2024-04-26T20:24:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.789+0000 7f93d98f8640 1 -- 172.21.15.37:0/1540338730 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f93d40cb4f0 con 0x7f93b406cbc0 2024-04-26T20:24:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.791+0000 7f93cbfff640 1 -- 172.21.15.37:0/1540338730 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f93d40cb4f0 con 0x7f93b406cbc0 2024-04-26T20:24:18.792 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-26T20:24:18.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T20:24:18.792 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephfs" 2024-04-26T20:24:18.792 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T20:24:18.792 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-26T20:24:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 -- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f93b406cbc0 msgr2=0x7f93b406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f93b406cbc0 0x7f93b406f080 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f93c4005c00 tx=0x7f93c4009040 comp rx=0 tx=0).stop 2024-04-26T20:24:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 -- 172.21.15.37:0/1540338730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d415f380 msgr2=0x7f93d414b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d415f380 0x7f93d414b240 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f93bc02a250 tx=0x7f93bc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 -- 172.21.15.37:0/1540338730 shutdown_connections 2024-04-26T20:24:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93d415f760 0x7f93d40c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d415f380 0x7f93d414b240 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f93b406cbc0 0x7f93b406f080 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 --2- 172.21.15.37:0/1540338730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d40d22e0 0x7f93d41627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 -- 172.21.15.37:0/1540338730 >> 172.21.15.37:0/1540338730 conn(0x7f93d409e560 msgr2=0x7f93d40b8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 -- 172.21.15.37:0/1540338730 shutdown_connections 2024-04-26T20:24:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.793+0000 7f93d98f8640 1 -- 172.21.15.37:0/1540338730 wait complete. 2024-04-26T20:24:18.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:24:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.907+0000 7f2129571640 1 Processor -- start 2024-04-26T20:24:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.907+0000 7f2129571640 1 -- start start 2024-04-26T20:24:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.908+0000 7f2129571640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f21240d2b30 0x7f21240cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.908+0000 7f2129571640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f212406e470 0x7f21240ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.908+0000 7f2129571640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21240d25f0 con 0x7f212415f5e0 2024-04-26T20:24:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.908+0000 7f2129571640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f21240cda70 con 0x7f21240d2b30 2024-04-26T20:24:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.908+0000 7f2129571640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f21240d2030 con 0x7f212406e470 2024-04-26T20:24:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.908+0000 7f21227fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f21240d2b30 0x7f21240cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.908+0000 7f21227fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f21240d2b30 0x7f21240cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52880/0 (socket says 172.21.15.37:52880) 2024-04-26T20:24:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.908+0000 7f21227fc640 1 -- 172.21.15.37:0/1364231140 learned_addr learned my addr 172.21.15.37:0/1364231140 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.908+0000 7f21237fe640 1 --2- 172.21.15.37:0/1364231140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f212406e470 0x7f21240ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.909+0000 7f21227fc640 1 -- 172.21.15.37:0/1364231140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f212406e470 msgr2=0x7f21240ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.909+0000 7f21227fc640 1 --2- 172.21.15.37:0/1364231140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f212406e470 0x7f21240ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.909+0000 7f21227fc640 1 -- 172.21.15.37:0/1364231140 >> v1:172.21.15.134:6789/0 conn(0x7f212415f5e0 legacy=0x7f212406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.909+0000 7f21227fc640 1 -- 172.21.15.37:0/1364231140 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21240d2310 con 0x7f21240d2b30 2024-04-26T20:24:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.909+0000 7f21227fc640 1 --2- 172.21.15.37:0/1364231140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21240d2b30 0x7f21240cdf00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f210c009870 tx=0x7f210c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe381ee1340a2896 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.910+0000 7f2121ffb640 1 -- 172.21.15.37:0/1364231140 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c037070 con 0x7f21240d2b30 2024-04-26T20:24:18.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.910+0000 7f2121ffb640 1 -- 172.21.15.37:0/1364231140 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f210c032440 con 0x7f21240d2b30 2024-04-26T20:24:18.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.910+0000 7f2121ffb640 1 -- 172.21.15.37:0/1364231140 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c032810 con 0x7f21240d2b30 2024-04-26T20:24:18.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.910+0000 7f2129571640 1 -- 172.21.15.37:0/1364231140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21240d2b30 msgr2=0x7f21240cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.910+0000 7f2129571640 1 --2- 172.21.15.37:0/1364231140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21240d2b30 0x7f21240cdf00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f210c009870 tx=0x7f210c02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.910+0000 7f2129571640 1 -- 172.21.15.37:0/1364231140 shutdown_connections 2024-04-26T20:24:18.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.910+0000 7f2129571640 1 --2- 172.21.15.37:0/1364231140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f212406e470 0x7f21240ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.910+0000 7f2129571640 1 --2- 172.21.15.37:0/1364231140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21240d2b30 0x7f21240cdf00 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.910+0000 7f2129571640 1 -- 172.21.15.37:0/1364231140 >> 172.21.15.37:0/1364231140 conn(0x7f21240d2f80 msgr2=0x7f21240d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:18.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.910+0000 7f2129571640 1 -- 172.21.15.37:0/1364231140 shutdown_connections 2024-04-26T20:24:18.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.910+0000 7f2129571640 1 -- 172.21.15.37:0/1364231140 wait complete. 2024-04-26T20:24:18.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.911+0000 7f2129571640 1 Processor -- start 2024-04-26T20:24:18.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.912+0000 7f2129571640 1 -- start start 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.912+0000 7f2129571640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f212406e470 0x7f212409cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.912+0000 7f2129571640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21240d2b30 0x7f21241581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.912+0000 7f2129571640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f212415f5e0 0x7f2124161b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.912+0000 7f2129571640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f212406d560 con 0x7f212406e470 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.912+0000 7f2129571640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21240d1cb0 con 0x7f212415f5e0 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.912+0000 7f2129571640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f212406d6e0 con 0x7f21240d2b30 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.912+0000 7f2122ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f212406e470 0x7f212409cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.912+0000 7f2122ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f212406e470 0x7f212409cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52888/0 (socket says 172.21.15.37:52888) 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.912+0000 7f2122ffd640 1 -- 172.21.15.37:0/1239069800 learned_addr learned my addr 172.21.15.37:0/1239069800 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f21227fc640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21240d2b30 0x7f21241581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f21237fe640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f212415f5e0 0x7f2124161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f2122ffd640 1 -- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21240d2b30 msgr2=0x7f21241581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f2122ffd640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21240d2b30 0x7f21241581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f2122ffd640 1 -- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f212415f5e0 msgr2=0x7f2124161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:18.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f2122ffd640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f212415f5e0 0x7f2124161b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f2122ffd640 1 -- 172.21.15.37:0/1239069800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212409d310 con 0x7f212406e470 2024-04-26T20:24:18.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f2122ffd640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f212406e470 0x7f212409cdd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f211400c790 tx=0x7f211400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:18.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f211b7fe640 1 -- 172.21.15.37:0/1239069800 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114016020 con 0x7f212406e470 2024-04-26T20:24:18.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f211b7fe640 1 -- 172.21.15.37:0/1239069800 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2114006d50 con 0x7f212406e470 2024-04-26T20:24:18.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f211b7fe640 1 -- 172.21.15.37:0/1239069800 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114003bb0 con 0x7f212406e470 2024-04-26T20:24:18.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f2129571640 1 -- 172.21.15.37:0/1239069800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2124159030 con 0x7f212406e470 2024-04-26T20:24:18.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.913+0000 7f2129571640 1 -- 172.21.15.37:0/1239069800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21241595f0 con 0x7f212406e470 2024-04-26T20:24:18.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.914+0000 7f2129571640 1 -- 172.21.15.37:0/1239069800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21240693e0 con 0x7f212406e470 2024-04-26T20:24:18.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.915+0000 7f211b7fe640 1 -- 172.21.15.37:0/1239069800 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f211400a850 con 0x7f212406e470 2024-04-26T20:24:18.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.915+0000 7f211b7fe640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f210406cbc0 0x7f210406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.915+0000 7f211b7fe640 1 -- 172.21.15.37:0/1239069800 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f211408bb10 con 0x7f212406e470 2024-04-26T20:24:18.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.915+0000 7f21227fc640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f210406cbc0 0x7f210406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:18.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.916+0000 7f21227fc640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f210406cbc0 0x7f210406f080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f210c034000 tx=0x7f210c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:18.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:18.920+0000 7f211b7fe640 1 -- 172.21.15.37:0/1239069800 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2114020020 con 0x7f212406e470 2024-04-26T20:24:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.105+0000 7f2129571640 1 -- 172.21.15.37:0/1239069800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f21240cfb90 con 0x7f212406e470 2024-04-26T20:24:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.105+0000 7f211b7fe640 1 -- 172.21.15.37:0/1239069800 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f2114059630 con 0x7f212406e470 2024-04-26T20:24:19.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 -- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f210406cbc0 msgr2=0x7f210406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:19.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f210406cbc0 0x7f210406f080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f210c034000 tx=0x7f210c02e000 comp rx=0 tx=0).stop 2024-04-26T20:24:19.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 -- 172.21.15.37:0/1239069800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f212406e470 msgr2=0x7f212409cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:19.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f212406e470 0x7f212409cdd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f211400c790 tx=0x7f211400cc50 comp rx=0 tx=0).stop 2024-04-26T20:24:19.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 -- 172.21.15.37:0/1239069800 shutdown_connections 2024-04-26T20:24:19.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f212415f5e0 0x7f2124161b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:19.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f210406cbc0 0x7f210406f080 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:19.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21240d2b30 0x7f21241581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:19.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 --2- 172.21.15.37:0/1239069800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f212406e470 0x7f212409cdd0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:19.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 -- 172.21.15.37:0/1239069800 >> 172.21.15.37:0/1239069800 conn(0x7f21240d2f80 msgr2=0x7f21240b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:19.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 -- 172.21.15.37:0/1239069800 shutdown_connections 2024-04-26T20:24:19.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.107+0000 7f2129571640 1 -- 172.21.15.37:0/1239069800 wait complete. 2024-04-26T20:24:19.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol76120 --mode=777 2024-04-26T20:24:19.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.222+0000 7f470686d640 1 Processor -- start 2024-04-26T20:24:19.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.222+0000 7f470686d640 1 -- start start 2024-04-26T20:24:19.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.223+0000 7f470686d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f470015f5d0 0x7f47000d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:19.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.223+0000 7f470686d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f47000d5640 0x7f47000b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:19.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.223+0000 7f470686d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47000b50f0 con 0x7f47000d4010 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.223+0000 7f470686d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f47000b5420 con 0x7f470015f5d0 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.223+0000 7f470686d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f47000d0a80 con 0x7f47000d5640 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.223+0000 7f46fffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f470015f5d0 0x7f47000d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.223+0000 7f46ff7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f47000d4010 0x7f47000d4f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47092/0 (socket says 172.21.15.37:47092) 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.223+0000 7f46ff7fe640 1 -- 172.21.15.37:0/1958104698 learned_addr learned my addr 172.21.15.37:0/1958104698 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.224+0000 7f4704e0a640 1 --2- 172.21.15.37:0/1958104698 >> v2:172.21.15.134:3300/0 conn(0x7f47000d5640 0x7f47000b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.224+0000 7f46fffff640 1 -- 172.21.15.37:0/1958104698 >> v2:172.21.15.134:3300/0 conn(0x7f47000d5640 msgr2=0x7f47000b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.224+0000 7f46fffff640 1 --2- 172.21.15.37:0/1958104698 >> v2:172.21.15.134:3300/0 conn(0x7f47000d5640 0x7f47000b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.224+0000 7f46fffff640 1 -- 172.21.15.37:0/1958104698 >> v1:172.21.15.37:6789/0 conn(0x7f47000d4010 legacy=0x7f47000d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.224+0000 7f46feffd640 1 -- 172.21.15.37:0/1958104698 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1904874766 0 0) 0x7f47000b50f0 con 0x7f47000d4010 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.224+0000 7f46fffff640 1 -- 172.21.15.37:0/1958104698 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47000d0d60 con 0x7f470015f5d0 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.224+0000 7f46fffff640 1 --2- 172.21.15.37:0/1958104698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470015f5d0 0x7f47000d3ad0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f47000cfb60 tx=0x7f46f002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fb04fac41c9b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.225+0000 7f46feffd640 1 -- 172.21.15.37:0/1958104698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0037070 con 0x7f470015f5d0 2024-04-26T20:24:19.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.225+0000 7f46feffd640 1 -- 172.21.15.37:0/1958104698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f0032440 con 0x7f470015f5d0 2024-04-26T20:24:19.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.225+0000 7f46feffd640 1 -- 172.21.15.37:0/1958104698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0032810 con 0x7f470015f5d0 2024-04-26T20:24:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.225+0000 7f470686d640 1 -- 172.21.15.37:0/1958104698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470015f5d0 msgr2=0x7f47000d3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.225+0000 7f470686d640 1 --2- 172.21.15.37:0/1958104698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470015f5d0 0x7f47000d3ad0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f47000cfb60 tx=0x7f46f002a230 comp rx=0 tx=0).stop 2024-04-26T20:24:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.225+0000 7f470686d640 1 -- 172.21.15.37:0/1958104698 shutdown_connections 2024-04-26T20:24:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.225+0000 7f470686d640 1 --2- 172.21.15.37:0/1958104698 >> v2:172.21.15.134:3300/0 conn(0x7f47000d5640 0x7f47000b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.225+0000 7f470686d640 1 --2- 172.21.15.37:0/1958104698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470015f5d0 0x7f47000d3ad0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.225+0000 7f470686d640 1 -- 172.21.15.37:0/1958104698 >> 172.21.15.37:0/1958104698 conn(0x7f470008ebd0 msgr2=0x7f470009e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.225+0000 7f470686d640 1 -- 172.21.15.37:0/1958104698 shutdown_connections 2024-04-26T20:24:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.225+0000 7f470686d640 1 -- 172.21.15.37:0/1958104698 wait complete. 2024-04-26T20:24:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.226+0000 7f470686d640 1 Processor -- start 2024-04-26T20:24:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.227+0000 7f470686d640 1 -- start start 2024-04-26T20:24:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.227+0000 7f470686d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47000d4010 0x7f470014dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.227+0000 7f470686d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47000d5640 0x7f47000ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.227+0000 7f470686d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470015f5d0 0x7f4700161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.227+0000 7f470686d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47000d34f0 con 0x7f47000d5640 2024-04-26T20:24:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.227+0000 7f470686d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f470015f9b0 con 0x7f470015f5d0 2024-04-26T20:24:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.227+0000 7f470686d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f47000b5df0 con 0x7f47000d4010 2024-04-26T20:24:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.227+0000 7f46ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47000d5640 0x7f47000ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.227+0000 7f46ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47000d5640 0x7f47000ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52908/0 (socket says 172.21.15.37:52908) 2024-04-26T20:24:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.227+0000 7f46ff7fe640 1 -- 172.21.15.37:0/1691957819 learned_addr learned my addr 172.21.15.37:0/1691957819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:24:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f4704e0a640 1 --2- 172.21.15.37:0/1691957819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470015f5d0 0x7f4700161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f46fffff640 1 --2- 172.21.15.37:0/1691957819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47000d4010 0x7f470014dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f46ff7fe640 1 -- 172.21.15.37:0/1691957819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47000d4010 msgr2=0x7f470014dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f46ff7fe640 1 --2- 172.21.15.37:0/1691957819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47000d4010 0x7f470014dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f46ff7fe640 1 -- 172.21.15.37:0/1691957819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470015f5d0 msgr2=0x7f4700161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f46ff7fe640 1 --2- 172.21.15.37:0/1691957819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470015f5d0 0x7f4700161e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f46ff7fe640 1 -- 172.21.15.37:0/1691957819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4700162910 con 0x7f47000d5640 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f4704e0a640 1 --2- 172.21.15.37:0/1691957819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470015f5d0 0x7f4700161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f46ff7fe640 1 --2- 172.21.15.37:0/1691957819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47000d5640 0x7f47000ca410 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c430 tx=0x7f46e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f46fcff9640 1 -- 172.21.15.37:0/1691957819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8017070 con 0x7f47000d5640 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f46fcff9640 1 -- 172.21.15.37:0/1691957819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e8005d00 con 0x7f47000d5640 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f46fcff9640 1 -- 172.21.15.37:0/1691957819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8007310 con 0x7f47000d5640 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f470686d640 1 -- 172.21.15.37:0/1691957819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470014cf00 con 0x7f47000d5640 2024-04-26T20:24:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.228+0000 7f470686d640 1 -- 172.21.15.37:0/1691957819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470014d7a0 con 0x7f47000d5640 2024-04-26T20:24:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.229+0000 7f470686d640 1 -- 172.21.15.37:0/1691957819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4700068510 con 0x7f47000d5640 2024-04-26T20:24:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.229+0000 7f46fcff9640 1 -- 172.21.15.37:0/1691957819 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f46e801a070 con 0x7f47000d5640 2024-04-26T20:24:19.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.230+0000 7f46fcff9640 1 --2- 172.21.15.37:0/1691957819 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f46e006cbc0 0x7f46e006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:19.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.230+0000 7f46fcff9640 1 -- 172.21.15.37:0/1691957819 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f46e808b4e0 con 0x7f47000d5640 2024-04-26T20:24:19.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.230+0000 7f46fffff640 1 --2- 172.21.15.37:0/1691957819 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f46e006cbc0 0x7f46e006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:19.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.231+0000 7f46fffff640 1 --2- 172.21.15.37:0/1691957819 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f46e006cbc0 0x7f46e006f080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f46f0034000 tx=0x7f46f002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:19.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.234+0000 7f46fcff9640 1 -- 172.21.15.37:0/1691957819 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4700068510 con 0x7f47000d5640 2024-04-26T20:24:19.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:24:19.446+0000 7f470686d640 1 -- 172.21.15.37:0/1691957819 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol76120", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f470014d9a0 con 0x7f46e006cbc0 2024-04-26T20:24:19.825 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:24:19.828 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:24:49.890 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:24:49.893 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:25:20.012 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:25:20.015 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:25:50.168 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:25:50.172 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:26:19.149 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-26T20:26:19.150 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:26:19.151 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:26:19.210 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:26:19.255 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:26:19.255 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:26:19.255 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:26:19.255 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:19.274 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:26:19.274 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:19.332 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:19.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.484+0000 7fc570f22640 1 Processor -- start 2024-04-26T20:26:19.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.484+0000 7fc570f22640 1 -- start start 2024-04-26T20:26:19.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.485+0000 7fc570f22640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc56c073120 0x7fc56c06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.485+0000 7fc570f22640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc56c15be00 con 0x7fc56c06f2c0 2024-04-26T20:26:19.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.485+0000 7fc570f22640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc56c15c0a0 con 0x7fc56c0d0540 2024-04-26T20:26:19.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.485+0000 7fc570f22640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc56c15c380 con 0x7fc56c073120 2024-04-26T20:26:19.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.485+0000 7fc569d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc56c073120 0x7fc56c06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.485+0000 7fc56ad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc56c06f2c0 0x7fc56c0cead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34358/0 (socket says 172.21.15.37:34358) 2024-04-26T20:26:19.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.485+0000 7fc56ad9d640 1 -- 172.21.15.37:0/1327857769 learned_addr learned my addr 172.21.15.37:0/1327857769 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:19.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.486+0000 7fc569d9b640 1 -- 172.21.15.37:0/1327857769 >> v1:172.21.15.134:6789/0 conn(0x7fc56c0d0540 legacy=0x7fc56c072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:19.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.486+0000 7fc569d9b640 1 -- 172.21.15.37:0/1327857769 >> v1:172.21.15.37:6789/0 conn(0x7fc56c06f2c0 legacy=0x7fc56c0cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.486+0000 7fc569d9b640 1 -- 172.21.15.37:0/1327857769 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56c15c660 con 0x7fc56c073120 2024-04-26T20:26:19.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.486+0000 7fc569d9b640 1 --2- 172.21.15.37:0/1327857769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc56c073120 0x7fc56c06ed80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc55400a940 tx=0x7fc55402b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76333851e0a1ff31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:19.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.487+0000 7fc56959a640 1 -- 172.21.15.37:0/1327857769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55400b040 con 0x7fc56c073120 2024-04-26T20:26:19.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.487+0000 7fc56959a640 1 -- 172.21.15.37:0/1327857769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc554030710 con 0x7fc56c073120 2024-04-26T20:26:19.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.487+0000 7fc56959a640 1 -- 172.21.15.37:0/1327857769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc554030ae0 con 0x7fc56c073120 2024-04-26T20:26:19.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.487+0000 7fc570f22640 1 -- 172.21.15.37:0/1327857769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc56c073120 msgr2=0x7fc56c06ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.487+0000 7fc570f22640 1 --2- 172.21.15.37:0/1327857769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc56c073120 0x7fc56c06ed80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc55400a940 tx=0x7fc55402b5a0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.487+0000 7fc570f22640 1 -- 172.21.15.37:0/1327857769 shutdown_connections 2024-04-26T20:26:19.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.487+0000 7fc570f22640 1 --2- 172.21.15.37:0/1327857769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc56c073120 0x7fc56c06ed80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.487+0000 7fc570f22640 1 -- 172.21.15.37:0/1327857769 >> 172.21.15.37:0/1327857769 conn(0x7fc56c0d0d50 msgr2=0x7fc56c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:19.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.487+0000 7fc570f22640 1 -- 172.21.15.37:0/1327857769 shutdown_connections 2024-04-26T20:26:19.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.487+0000 7fc570f22640 1 -- 172.21.15.37:0/1327857769 wait complete. 2024-04-26T20:26:19.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.489+0000 7fc570f22640 1 Processor -- start 2024-04-26T20:26:19.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.489+0000 7fc570f22640 1 -- start start 2024-04-26T20:26:19.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc570f22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc56c06f2c0 0x7fc56c1522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc570f22640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc56c073120 0x7fc56c1f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc570f22640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c0d0540 0x7fc56c1f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc570f22640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c0d0aa0 con 0x7fc56c06f2c0 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc570f22640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c0d0920 con 0x7fc56c0d0540 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc570f22640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc56c06e250 con 0x7fc56c073120 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc56a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc56c06f2c0 0x7fc56c1522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc56a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc56c06f2c0 0x7fc56c1522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40482/0 (socket says 172.21.15.37:40482) 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc56a59c640 1 -- 172.21.15.37:0/2282269313 learned_addr learned my addr 172.21.15.37:0/2282269313 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc569d9b640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc56c073120 0x7fc56c1f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc56ad9d640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c0d0540 0x7fc56c1f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc56a59c640 1 -- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc56c073120 msgr2=0x7fc56c1f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc56a59c640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc56c073120 0x7fc56c1f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc56a59c640 1 -- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c0d0540 msgr2=0x7fc56c1f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc56a59c640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c0d0540 0x7fc56c1f2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc56a59c640 1 -- 172.21.15.37:0/2282269313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56c1529d0 con 0x7fc56c06f2c0 2024-04-26T20:26:19.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc569d9b640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc56c073120 0x7fc56c1f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:19.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc56ad9d640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c0d0540 0x7fc56c1f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:19.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc56a59c640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc56c06f2c0 0x7fc56c1522f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b3f0 tx=0x7fc56400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:19.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc562ffd640 1 -- 172.21.15.37:0/2282269313 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564015020 con 0x7fc56c06f2c0 2024-04-26T20:26:19.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc562ffd640 1 -- 172.21.15.37:0/2282269313 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc564011830 con 0x7fc56c06f2c0 2024-04-26T20:26:19.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc570f22640 1 -- 172.21.15.37:0/2282269313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56c152cb0 con 0x7fc56c06f2c0 2024-04-26T20:26:19.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc562ffd640 1 -- 172.21.15.37:0/2282269313 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564011c20 con 0x7fc56c06f2c0 2024-04-26T20:26:19.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.490+0000 7fc570f22640 1 -- 172.21.15.37:0/2282269313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc56c153270 con 0x7fc56c06f2c0 2024-04-26T20:26:19.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.492+0000 7fc570f22640 1 -- 172.21.15.37:0/2282269313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc56c06a360 con 0x7fc56c06f2c0 2024-04-26T20:26:19.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.492+0000 7fc562ffd640 1 -- 172.21.15.37:0/2282269313 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fc564018020 con 0x7fc56c06f2c0 2024-04-26T20:26:19.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.492+0000 7fc562ffd640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc54c06cbc0 0x7fc54c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.493+0000 7fc562ffd640 1 -- 172.21.15.37:0/2282269313 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fc56408bd50 con 0x7fc56c06f2c0 2024-04-26T20:26:19.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.493+0000 7fc569d9b640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc54c06cbc0 0x7fc54c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.493+0000 7fc569d9b640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc54c06cbc0 0x7fc54c06f080 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc554035000 tx=0x7fc55400c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:19.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.497+0000 7fc562ffd640 1 -- 172.21.15.37:0/2282269313 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc56401f030 con 0x7fc56c06f2c0 2024-04-26T20:26:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.684+0000 7fc570f22640 1 -- 172.21.15.37:0/2282269313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc56c06f8e0 con 0x7fc56c06f2c0 2024-04-26T20:26:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.685+0000 7fc562ffd640 1 -- 172.21.15.37:0/2282269313 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16715 (secure 0 0 0) 0x7fc5640172c0 con 0x7fc56c06f2c0 2024-04-26T20:26:19.687 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:19.687 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":48,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:24:31.476288+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:24:08.047629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:24:08.708738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.f","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:19.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.687+0000 7fc570f22640 1 -- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc54c06cbc0 msgr2=0x7fc54c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.687+0000 7fc570f22640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc54c06cbc0 0x7fc54c06f080 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc554035000 tx=0x7fc55400c040 comp rx=0 tx=0).stop 2024-04-26T20:26:19.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.687+0000 7fc570f22640 1 -- 172.21.15.37:0/2282269313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc56c06f2c0 msgr2=0x7fc56c1522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.687+0000 7fc570f22640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc56c06f2c0 0x7fc56c1522f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b3f0 tx=0x7fc56400b8b0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.687+0000 7fc570f22640 1 -- 172.21.15.37:0/2282269313 shutdown_connections 2024-04-26T20:26:19.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.687+0000 7fc570f22640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c0d0540 0x7fc56c1f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.687+0000 7fc570f22640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fc54c06cbc0 0x7fc54c06f080 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.687+0000 7fc570f22640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc56c073120 0x7fc56c1f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.687+0000 7fc570f22640 1 --2- 172.21.15.37:0/2282269313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc56c06f2c0 0x7fc56c1522f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.687+0000 7fc570f22640 1 -- 172.21.15.37:0/2282269313 >> 172.21.15.37:0/2282269313 conn(0x7fc56c0d0d50 msgr2=0x7fc56c14c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:19.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.688+0000 7fc570f22640 1 -- 172.21.15.37:0/2282269313 shutdown_connections 2024-04-26T20:26:19.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.688+0000 7fc570f22640 1 -- 172.21.15.37:0/2282269313 wait complete. 2024-04-26T20:26:19.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:24:08.047629+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:24:08.708738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:26:19.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:26:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.802+0000 7f1225dc9640 1 Processor -- start 2024-04-26T20:26:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.803+0000 7f1225dc9640 1 -- start start 2024-04-26T20:26:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.803+0000 7f1225dc9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1220160800 0x7f12201610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.803+0000 7f1225dc9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f12200d5b60 0x7f12200d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.803+0000 7f1225dc9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12200d0930 con 0x7f12200d5f80 2024-04-26T20:26:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.803+0000 7f1225dc9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f12200d0c00 con 0x7f12200d5b60 2024-04-26T20:26:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.803+0000 7f1225dc9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f12200d0ee0 con 0x7f1220160800 2024-04-26T20:26:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.803+0000 7f121ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f12200d5f80 0x7f12200d3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34364/0 (socket says 172.21.15.37:34364) 2024-04-26T20:26:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.803+0000 7f121ffff640 1 -- 172.21.15.37:0/4004538477 learned_addr learned my addr 172.21.15.37:0/4004538477 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.803+0000 7f121effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f12200d5b60 0x7f12200d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.804+0000 7f121f7fe640 1 --2- 172.21.15.37:0/4004538477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1220160800 0x7f12201610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.804+0000 7f121effd640 1 -- 172.21.15.37:0/4004538477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1220160800 msgr2=0x7f12201610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:19.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.804+0000 7f121effd640 1 --2- 172.21.15.37:0/4004538477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1220160800 0x7f12201610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.804+0000 7f121effd640 1 -- 172.21.15.37:0/4004538477 >> v1:172.21.15.37:6789/0 conn(0x7f12200d5f80 legacy=0x7f12200d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.804+0000 7f121e7fc640 1 -- 172.21.15.37:0/4004538477 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1036140749 0 0) 0x7f12200d0930 con 0x7f12200d5f80 2024-04-26T20:26:19.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.804+0000 7f121effd640 1 -- 172.21.15.37:0/4004538477 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12200d11c0 con 0x7f12200d5b60 2024-04-26T20:26:19.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.804+0000 7f121f7fe640 1 --2- 172.21.15.37:0/4004538477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1220160800 0x7f12201610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f121effd640 1 --2- 172.21.15.37:0/4004538477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d5b60 0x7f12200d1c10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f12140097a0 tx=0x7f121402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2cbfd2551b9db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f121e7fc640 1 -- 172.21.15.37:0/4004538477 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214037070 con 0x7f12200d5b60 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f121e7fc640 1 -- 172.21.15.37:0/4004538477 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1214032440 con 0x7f12200d5b60 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f121e7fc640 1 -- 172.21.15.37:0/4004538477 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214032810 con 0x7f12200d5b60 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f1225dc9640 1 -- 172.21.15.37:0/4004538477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d5b60 msgr2=0x7f12200d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f1225dc9640 1 --2- 172.21.15.37:0/4004538477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d5b60 0x7f12200d1c10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f12140097a0 tx=0x7f121402a180 comp rx=0 tx=0).stop 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f1225dc9640 1 -- 172.21.15.37:0/4004538477 shutdown_connections 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f1225dc9640 1 --2- 172.21.15.37:0/4004538477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d5b60 0x7f12200d1c10 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f1225dc9640 1 --2- 172.21.15.37:0/4004538477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1220160800 0x7f12201610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f1225dc9640 1 -- 172.21.15.37:0/4004538477 >> 172.21.15.37:0/4004538477 conn(0x7f122009e550 msgr2=0x7f12200d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f1225dc9640 1 -- 172.21.15.37:0/4004538477 shutdown_connections 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.805+0000 7f1225dc9640 1 -- 172.21.15.37:0/4004538477 wait complete. 2024-04-26T20:26:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.806+0000 7f1225dc9640 1 Processor -- start 2024-04-26T20:26:19.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.807+0000 7f1225dc9640 1 -- start start 2024-04-26T20:26:19.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.807+0000 7f1225dc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d5b60 0x7f12200ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.807+0000 7f1225dc9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1220160800 0x7f12200ba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.807+0000 7f1225dc9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12201f5630 0x7f12201596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.807+0000 7f1225dc9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f12200b6300 con 0x7f12200d5b60 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.807+0000 7f1225dc9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12200d30f0 con 0x7f12201f5630 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.807+0000 7f1225dc9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1220160be0 con 0x7f1220160800 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.807+0000 7f121f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d5b60 0x7f12200ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f121f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d5b60 0x7f12200ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40498/0 (socket says 172.21.15.37:40498) 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f121f7fe640 1 -- 172.21.15.37:0/238692987 learned_addr learned my addr 172.21.15.37:0/238692987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f121f7fe640 1 -- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1220160800 msgr2=0x7f12200ba8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f121f7fe640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1220160800 0x7f12200ba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f121f7fe640 1 -- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12201f5630 msgr2=0x7f12201596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f121f7fe640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12201f5630 0x7f12201596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f121f7fe640 1 -- 172.21.15.37:0/238692987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12200ce610 con 0x7f12200d5b60 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f121f7fe640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d5b60 0x7f12200ccd20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f121000c7f0 tx=0x7f121000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f1207fff640 1 -- 172.21.15.37:0/238692987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210015070 con 0x7f12200d5b60 2024-04-26T20:26:19.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f1207fff640 1 -- 172.21.15.37:0/238692987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1210007820 con 0x7f12200d5b60 2024-04-26T20:26:19.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f1207fff640 1 -- 172.21.15.37:0/238692987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210007bd0 con 0x7f12200d5b60 2024-04-26T20:26:19.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f1225dc9640 1 -- 172.21.15.37:0/238692987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12200ce890 con 0x7f12200d5b60 2024-04-26T20:26:19.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.808+0000 7f1225dc9640 1 -- 172.21.15.37:0/238692987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12201f6140 con 0x7f12200d5b60 2024-04-26T20:26:19.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.809+0000 7f1225dc9640 1 -- 172.21.15.37:0/238692987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12200bade0 con 0x7f12200d5b60 2024-04-26T20:26:19.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.810+0000 7f1207fff640 1 -- 172.21.15.37:0/238692987 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f1210018070 con 0x7f12200d5b60 2024-04-26T20:26:19.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.810+0000 7f1207fff640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f120c06cbc0 0x7f120c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.810+0000 7f1207fff640 1 -- 172.21.15.37:0/238692987 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f121008c1d0 con 0x7f12200d5b60 2024-04-26T20:26:19.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.810+0000 7f121effd640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f120c06cbc0 0x7f120c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.811+0000 7f121effd640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f120c06cbc0 0x7f120c06f080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1214034000 tx=0x7f121402c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.814+0000 7f1207fff640 1 -- 172.21.15.37:0/238692987 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f12200bade0 con 0x7f12200d5b60 2024-04-26T20:26:19.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.997+0000 7f1225dc9640 1 -- 172.21.15.37:0/238692987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f122006a360 con 0x7f12200d5b60 2024-04-26T20:26:19.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.997+0000 7f1207fff640 1 -- 172.21.15.37:0/238692987 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f12200ce610 con 0x7f12200d5b60 2024-04-26T20:26:19.998 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:26:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 -- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f120c06cbc0 msgr2=0x7f120c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f120c06cbc0 0x7f120c06f080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1214034000 tx=0x7f121402c000 comp rx=0 tx=0).stop 2024-04-26T20:26:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 -- 172.21.15.37:0/238692987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d5b60 msgr2=0x7f12200ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d5b60 0x7f12200ccd20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f121000c7f0 tx=0x7f121000ccb0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 -- 172.21.15.37:0/238692987 shutdown_connections 2024-04-26T20:26:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12201f5630 0x7f12201596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f120c06cbc0 0x7f120c06f080 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1220160800 0x7f12200ba8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 --2- 172.21.15.37:0/238692987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d5b60 0x7f12200ccd20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 -- 172.21.15.37:0/238692987 >> 172.21.15.37:0/238692987 conn(0x7f122009e550 msgr2=0x7f12200d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 -- 172.21.15.37:0/238692987 shutdown_connections 2024-04-26T20:26:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:19.999+0000 7f1225dc9640 1 -- 172.21.15.37:0/238692987 wait complete. 2024-04-26T20:26:20.011 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:26:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.113+0000 7f8cba967640 1 Processor -- start 2024-04-26T20:26:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.113+0000 7f8cba967640 1 -- start start 2024-04-26T20:26:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.114+0000 7f8cba967640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cb40d2b30 0x7f8cb40cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.114+0000 7f8cba967640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8cb406e470 0x7f8cb40ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.114+0000 7f8cba967640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cb40d25f0 con 0x7f8cb415f5e0 2024-04-26T20:26:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.114+0000 7f8cba967640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f8cb40cda70 con 0x7f8cb406e470 2024-04-26T20:26:20.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.114+0000 7f8cba967640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb40d2030 con 0x7f8cb40d2b30 2024-04-26T20:26:20.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.115+0000 7f8cb8f04640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8cb406e470 0x7f8cb40ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.115+0000 7f8cb37fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cb40d2b30 0x7f8cb40cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.115+0000 7f8cb8f04640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8cb406e470 0x7f8cb40ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:43540/0 (socket says 172.21.15.37:43540) 2024-04-26T20:26:20.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.115+0000 7f8cb8f04640 1 -- 172.21.15.37:0/2796527369 learned_addr learned my addr 172.21.15.37:0/2796527369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:20.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.115+0000 7f8cb3fff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f8cb415f5e0 0x7f8cb406dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:48644/0 (socket says 172.21.15.37:48644) 2024-04-26T20:26:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.115+0000 7f8cb8f04640 1 -- 172.21.15.37:0/2796527369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cb40d2b30 msgr2=0x7f8cb40cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.115+0000 7f8cb8f04640 1 --2- 172.21.15.37:0/2796527369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cb40d2b30 0x7f8cb40cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.115+0000 7f8cb8f04640 1 -- 172.21.15.37:0/2796527369 >> v1:172.21.15.134:6789/0 conn(0x7f8cb415f5e0 legacy=0x7f8cb406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.115+0000 7f8cb8f04640 1 -- 172.21.15.37:0/2796527369 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb40d2310 con 0x7f8cb406e470 2024-04-26T20:26:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.115+0000 7f8cb2ffd640 1 -- 172.21.15.37:0/2796527369 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 632604852 0 0) 0x7f8cb40d25f0 con 0x7f8cb415f5e0 2024-04-26T20:26:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.116+0000 7f8cb8f04640 1 --2- 172.21.15.37:0/2796527369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb406e470 0x7f8cb40ce440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009e50 tx=0x7f8cac02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b2cd45d8ba1fb86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.116+0000 7f8cb2ffd640 1 -- 172.21.15.37:0/2796527369 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac036070 con 0x7f8cb406e470 2024-04-26T20:26:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.116+0000 7f8cb2ffd640 1 -- 172.21.15.37:0/2796527369 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cac031600 con 0x7f8cb406e470 2024-04-26T20:26:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.116+0000 7f8cb2ffd640 1 -- 172.21.15.37:0/2796527369 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac0319d0 con 0x7f8cb406e470 2024-04-26T20:26:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.116+0000 7f8cba967640 1 -- 172.21.15.37:0/2796527369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb406e470 msgr2=0x7f8cb40ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.116+0000 7f8cba967640 1 --2- 172.21.15.37:0/2796527369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb406e470 0x7f8cb40ce440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009e50 tx=0x7f8cac02a680 comp rx=0 tx=0).stop 2024-04-26T20:26:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.116+0000 7f8cba967640 1 -- 172.21.15.37:0/2796527369 shutdown_connections 2024-04-26T20:26:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.116+0000 7f8cba967640 1 --2- 172.21.15.37:0/2796527369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb406e470 0x7f8cb40ce440 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.116+0000 7f8cba967640 1 --2- 172.21.15.37:0/2796527369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cb40d2b30 0x7f8cb40cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.116+0000 7f8cba967640 1 -- 172.21.15.37:0/2796527369 >> 172.21.15.37:0/2796527369 conn(0x7f8cb40d2f80 msgr2=0x7f8cb40d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.116+0000 7f8cba967640 1 -- 172.21.15.37:0/2796527369 shutdown_connections 2024-04-26T20:26:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.117+0000 7f8cba967640 1 -- 172.21.15.37:0/2796527369 wait complete. 2024-04-26T20:26:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.118+0000 7f8cba967640 1 Processor -- start 2024-04-26T20:26:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.118+0000 7f8cba967640 1 -- start start 2024-04-26T20:26:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.118+0000 7f8cba967640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cb406e470 0x7f8cb414b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.118+0000 7f8cba967640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb40d2b30 0x7f8cb415acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.118+0000 7f8cba967640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cb415f5e0 0x7f8cb414bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.118+0000 7f8cba967640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb406d560 con 0x7f8cb415f5e0 2024-04-26T20:26:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.118+0000 7f8cba967640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb40d1cb0 con 0x7f8cb40d2b30 2024-04-26T20:26:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.118+0000 7f8cba967640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb406d6e0 con 0x7f8cb406e470 2024-04-26T20:26:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb8f04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cb415f5e0 0x7f8cb414bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb8f04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cb415f5e0 0x7f8cb414bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40514/0 (socket says 172.21.15.37:40514) 2024-04-26T20:26:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb8f04640 1 -- 172.21.15.37:0/840173606 learned_addr learned my addr 172.21.15.37:0/840173606 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb3fff640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cb406e470 0x7f8cb414b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb37fe640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb40d2b30 0x7f8cb415acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb8f04640 1 -- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cb406e470 msgr2=0x7f8cb414b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb8f04640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cb406e470 0x7f8cb414b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb8f04640 1 -- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb40d2b30 msgr2=0x7f8cb415acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb8f04640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb40d2b30 0x7f8cb415acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb8f04640 1 -- 172.21.15.37:0/840173606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb4159ef0 con 0x7f8cb415f5e0 2024-04-26T20:26:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb37fe640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb40d2b30 0x7f8cb415acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb3fff640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cb406e470 0x7f8cb414b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb8f04640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cb415f5e0 0x7f8cb414bdf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8cac034040 tx=0x7f8cac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb0ff9640 1 -- 172.21.15.37:0/840173606 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac041020 con 0x7f8cb415f5e0 2024-04-26T20:26:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb0ff9640 1 -- 172.21.15.37:0/840173606 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cac037040 con 0x7f8cb415f5e0 2024-04-26T20:26:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cba967640 1 -- 172.21.15.37:0/840173606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb415a1d0 con 0x7f8cb415f5e0 2024-04-26T20:26:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cb0ff9640 1 -- 172.21.15.37:0/840173606 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac036050 con 0x7f8cb415f5e0 2024-04-26T20:26:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.119+0000 7f8cba967640 1 -- 172.21.15.37:0/840173606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb415a790 con 0x7f8cb415f5e0 2024-04-26T20:26:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.120+0000 7f8cba967640 1 -- 172.21.15.37:0/840173606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb40693e0 con 0x7f8cb415f5e0 2024-04-26T20:26:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.121+0000 7f8cb0ff9640 1 -- 172.21.15.37:0/840173606 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f8cac007bd0 con 0x7f8cb415f5e0 2024-04-26T20:26:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.121+0000 7f8cb0ff9640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8ca006cbc0 0x7f8ca006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.121+0000 7f8cb0ff9640 1 -- 172.21.15.37:0/840173606 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f8cac0ac760 con 0x7f8cb415f5e0 2024-04-26T20:26:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.122+0000 7f8cb3fff640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8ca006cbc0 0x7f8ca006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.122+0000 7f8cb3fff640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8ca006cbc0 0x7f8ca006f080 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4000f60 tx=0x7f8ca4007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:20.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.126+0000 7f8cb0ff9640 1 -- 172.21.15.37:0/840173606 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8cac04b060 con 0x7f8cb415f5e0 2024-04-26T20:26:20.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:26:20.278 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:26:20.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.310+0000 7f8cba967640 1 -- 172.21.15.37:0/840173606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8cb40cf3d0 con 0x7f8cb415f5e0 2024-04-26T20:26:20.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.310+0000 7f8cb0ff9640 1 -- 172.21.15.37:0/840173606 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) ==== 104+0+11 (secure 0 0 0) 0x7f8cac07a780 con 0x7f8cb415f5e0 2024-04-26T20:26:20.311 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:26:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 -- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8ca006cbc0 msgr2=0x7f8ca006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8ca006cbc0 0x7f8ca006f080 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4000f60 tx=0x7f8ca4007ca0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 -- 172.21.15.37:0/840173606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cb415f5e0 msgr2=0x7f8cb414bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cb415f5e0 0x7f8cb414bdf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8cac034040 tx=0x7f8cac004a40 comp rx=0 tx=0).stop 2024-04-26T20:26:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 -- 172.21.15.37:0/840173606 shutdown_connections 2024-04-26T20:26:20.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cb415f5e0 0x7f8cb414bdf0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb40d2b30 0x7f8cb415acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8ca006cbc0 0x7f8ca006f080 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 --2- 172.21.15.37:0/840173606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cb406e470 0x7f8cb414b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 -- 172.21.15.37:0/840173606 >> 172.21.15.37:0/840173606 conn(0x7f8cb40d2f80 msgr2=0x7f8cb40d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:20.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 -- 172.21.15.37:0/840173606 shutdown_connections 2024-04-26T20:26:20.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.312+0000 7f8cba967640 1 -- 172.21.15.37:0/840173606 wait complete. 2024-04-26T20:26:20.324 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:26:20.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.473+0000 7fdcf4a72640 1 Processor -- start 2024-04-26T20:26:20.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.473+0000 7fdcf4a72640 1 -- start start 2024-04-26T20:26:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.474+0000 7fdcf4a72640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdcf006c950 0x7fdcf00cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.474+0000 7fdcf4a72640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf0074050 con 0x7fdcf015f3c0 2024-04-26T20:26:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.474+0000 7fdcf4a72640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf00b1060 con 0x7fdcf00d1990 2024-04-26T20:26:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.474+0000 7fdcf4a72640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf00b1340 con 0x7fdcf006c950 2024-04-26T20:26:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.474+0000 7fdcee59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdcf015f3c0 0x7fdcf006c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34372/0 (socket says 172.21.15.37:34372) 2024-04-26T20:26:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.474+0000 7fdcee59c640 1 -- 172.21.15.37:0/1433391785 learned_addr learned my addr 172.21.15.37:0/1433391785 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.474+0000 7fdcedd9b640 1 --2- 172.21.15.37:0/1433391785 >> v2:172.21.15.134:3300/0 conn(0x7fdcf006c950 0x7fdcf00cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.475+0000 7fdced59a640 1 -- 172.21.15.37:0/1433391785 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2454420104 0 0) 0x7fdcf0074050 con 0x7fdcf015f3c0 2024-04-26T20:26:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.475+0000 7fdced59a640 1 -- 172.21.15.37:0/1433391785 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcdc003520 con 0x7fdcf015f3c0 2024-04-26T20:26:20.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.475+0000 7fdcedd9b640 1 -- 172.21.15.37:0/1433391785 >> v1:172.21.15.134:6789/0 conn(0x7fdcf00d1990 legacy=0x7fdcf00cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.475+0000 7fdcedd9b640 1 -- 172.21.15.37:0/1433391785 >> v1:172.21.15.37:6789/0 conn(0x7fdcf015f3c0 legacy=0x7fdcf006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.475+0000 7fdced59a640 1 -- 172.21.15.37:0/1433391785 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4085385070 0 0) 0x7fdcdc003520 con 0x7fdcf015f3c0 2024-04-26T20:26:20.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.475+0000 7fdcedd9b640 1 -- 172.21.15.37:0/1433391785 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf00b1620 con 0x7fdcf006c950 2024-04-26T20:26:20.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.476+0000 7fdcedd9b640 1 --2- 172.21.15.37:0/1433391785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcf006c950 0x7fdcf00cc4d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80097a0 tx=0x7fdcd802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87e1e347de9e30a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:20.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.476+0000 7fdced59a640 1 -- 172.21.15.37:0/1433391785 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8037070 con 0x7fdcf006c950 2024-04-26T20:26:20.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.476+0000 7fdced59a640 1 -- 172.21.15.37:0/1433391785 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd802abc0 con 0x7fdcf006c950 2024-04-26T20:26:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.476+0000 7fdced59a640 1 -- 172.21.15.37:0/1433391785 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8001670 con 0x7fdcf006c950 2024-04-26T20:26:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.476+0000 7fdcf4a72640 1 -- 172.21.15.37:0/1433391785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcf006c950 msgr2=0x7fdcf00cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.476+0000 7fdcf4a72640 1 --2- 172.21.15.37:0/1433391785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcf006c950 0x7fdcf00cc4d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80097a0 tx=0x7fdcd802a130 comp rx=0 tx=0).stop 2024-04-26T20:26:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.476+0000 7fdcf4a72640 1 -- 172.21.15.37:0/1433391785 shutdown_connections 2024-04-26T20:26:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.476+0000 7fdcf4a72640 1 --2- 172.21.15.37:0/1433391785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcf006c950 0x7fdcf00cc4d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.476+0000 7fdcf4a72640 1 -- 172.21.15.37:0/1433391785 >> 172.21.15.37:0/1433391785 conn(0x7fdcf00d1e80 msgr2=0x7fdcf00d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.476+0000 7fdcf4a72640 1 -- 172.21.15.37:0/1433391785 shutdown_connections 2024-04-26T20:26:20.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.477+0000 7fdcf4a72640 1 -- 172.21.15.37:0/1433391785 wait complete. 2024-04-26T20:26:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.478+0000 7fdcf4a72640 1 Processor -- start 2024-04-26T20:26:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.478+0000 7fdcf4a72640 1 -- start start 2024-04-26T20:26:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcf4a72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcf006c950 0x7fdcf00b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcf4a72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcf00d1990 0x7fdcf0150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcf4a72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcf015f3c0 0x7fdcf0150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcf4a72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf00d1530 con 0x7fdcf00d1990 2024-04-26T20:26:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcf4a72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf00d13b0 con 0x7fdcf006c950 2024-04-26T20:26:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcf4a72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf015f7a0 con 0x7fdcf015f3c0 2024-04-26T20:26:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcedd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcf00d1990 0x7fdcf0150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcedd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcf00d1990 0x7fdcf0150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40524/0 (socket says 172.21.15.37:40524) 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcedd9b640 1 -- 172.21.15.37:0/579440115 learned_addr learned my addr 172.21.15.37:0/579440115 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcee59c640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcf006c950 0x7fdcf00b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcedd9b640 1 -- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcf015f3c0 msgr2=0x7fdcf0150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdceed9d640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcf015f3c0 0x7fdcf0150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcedd9b640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcf015f3c0 0x7fdcf0150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcedd9b640 1 -- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcf006c950 msgr2=0x7fdcf00b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcedd9b640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcf006c950 0x7fdcf00b70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcedd9b640 1 -- 172.21.15.37:0/579440115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf01510f0 con 0x7fdcf00d1990 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdcedd9b640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcf00d1990 0x7fdcf0150bb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8035040 tx=0x7fdcd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdce6ffd640 1 -- 172.21.15.37:0/579440115 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8042020 con 0x7fdcf00d1990 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdce6ffd640 1 -- 172.21.15.37:0/579440115 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd8003a40 con 0x7fdcf00d1990 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.479+0000 7fdce6ffd640 1 -- 172.21.15.37:0/579440115 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8037040 con 0x7fdcf00d1990 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.480+0000 7fdcf4a72640 1 -- 172.21.15.37:0/579440115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf0151390 con 0x7fdcf00d1990 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.480+0000 7fdcf4a72640 1 -- 172.21.15.37:0/579440115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf0151950 con 0x7fdcf00d1990 2024-04-26T20:26:20.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.480+0000 7fdcf4a72640 1 -- 172.21.15.37:0/579440115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf006a360 con 0x7fdcf00d1990 2024-04-26T20:26:20.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.481+0000 7fdce6ffd640 1 -- 172.21.15.37:0/579440115 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fdcd8045020 con 0x7fdcf00d1990 2024-04-26T20:26:20.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.481+0000 7fdce6ffd640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdcd006cbc0 0x7fdcd006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.482+0000 7fdce6ffd640 1 -- 172.21.15.37:0/579440115 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fdcd80ac460 con 0x7fdcf00d1990 2024-04-26T20:26:20.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.482+0000 7fdcee59c640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdcd006cbc0 0x7fdcd006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.482+0000 7fdcee59c640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdcd006cbc0 0x7fdcd006f080 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdce80011e0 tx=0x7fdce8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:20.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.486+0000 7fdce6ffd640 1 -- 172.21.15.37:0/579440115 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdcd804c070 con 0x7fdcf00d1990 2024-04-26T20:26:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.673+0000 7fdcf4a72640 1 -- 172.21.15.37:0/579440115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdcf00b7e00 con 0x7fdcf00d1990 2024-04-26T20:26:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.673+0000 7fdce6ffd640 1 -- 172.21.15.37:0/579440115 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) ==== 100+0+11 (secure 0 0 0) 0x7fdcd807a420 con 0x7fdcf00d1990 2024-04-26T20:26:20.675 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:26:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 -- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdcd006cbc0 msgr2=0x7fdcd006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdcd006cbc0 0x7fdcd006f080 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdce80011e0 tx=0x7fdce8009040 comp rx=0 tx=0).stop 2024-04-26T20:26:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 -- 172.21.15.37:0/579440115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcf00d1990 msgr2=0x7fdcf0150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcf00d1990 0x7fdcf0150bb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8035040 tx=0x7fdcd80046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 -- 172.21.15.37:0/579440115 shutdown_connections 2024-04-26T20:26:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdcd006cbc0 0x7fdcd006f080 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcf015f3c0 0x7fdcf0150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcf00d1990 0x7fdcf0150bb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 --2- 172.21.15.37:0/579440115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcf006c950 0x7fdcf00b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 -- 172.21.15.37:0/579440115 >> 172.21.15.37:0/579440115 conn(0x7fdcf00d1e80 msgr2=0x7fdcf00b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 -- 172.21.15.37:0/579440115 shutdown_connections 2024-04-26T20:26:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.676+0000 7fdcf4a72640 1 -- 172.21.15.37:0/579440115 wait complete. 2024-04-26T20:26:20.688 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:26:20.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.788+0000 7f9deba1d640 1 Processor -- start 2024-04-26T20:26:20.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.789+0000 7f9deba1d640 1 -- start start 2024-04-26T20:26:20.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.789+0000 7f9deba1d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de409ba50 0x7f9de40b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.789+0000 7f9deba1d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9de406d560 0x7f9de409be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.789+0000 7f9deba1d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de40cd650 con 0x7f9de415ebd0 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.789+0000 7f9deba1d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f9de409c370 con 0x7f9de406d560 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.789+0000 7f9deba1d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9de40cd8f0 con 0x7f9de409ba50 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.789+0000 7f9de8fb8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9de406d560 0x7f9de409be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.789+0000 7f9de9fba640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9de415ebd0 0x7f9de415f050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34382/0 (socket says 172.21.15.37:34382) 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.790+0000 7f9de9fba640 1 -- 172.21.15.37:0/1618968439 learned_addr learned my addr 172.21.15.37:0/1618968439 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.790+0000 7f9de97b9640 1 --2- 172.21.15.37:0/1618968439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de409ba50 0x7f9de40b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.790+0000 7f9de8fb8640 1 -- 172.21.15.37:0/1618968439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de409ba50 msgr2=0x7f9de40b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.790+0000 7f9de8fb8640 1 --2- 172.21.15.37:0/1618968439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de409ba50 0x7f9de40b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.790+0000 7f9de8fb8640 1 -- 172.21.15.37:0/1618968439 >> v1:172.21.15.37:6789/0 conn(0x7f9de415ebd0 legacy=0x7f9de415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.790+0000 7f9de8fb8640 1 -- 172.21.15.37:0/1618968439 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de40cdbd0 con 0x7f9de406d560 2024-04-26T20:26:20.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.790+0000 7f9de8fb8640 1 --2- 172.21.15.37:0/1618968439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de406d560 0x7f9de409be30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80097a0 tx=0x7f9dd802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa28b6df255d0dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:20.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.791+0000 7f9de3fff640 1 -- 172.21.15.37:0/1618968439 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8037070 con 0x7f9de406d560 2024-04-26T20:26:20.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.791+0000 7f9de3fff640 1 -- 172.21.15.37:0/1618968439 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd8032440 con 0x7f9de406d560 2024-04-26T20:26:20.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.791+0000 7f9de3fff640 1 -- 172.21.15.37:0/1618968439 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8032810 con 0x7f9de406d560 2024-04-26T20:26:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.791+0000 7f9deba1d640 1 -- 172.21.15.37:0/1618968439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de406d560 msgr2=0x7f9de409be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.791+0000 7f9deba1d640 1 --2- 172.21.15.37:0/1618968439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de406d560 0x7f9de409be30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80097a0 tx=0x7f9dd802a180 comp rx=0 tx=0).stop 2024-04-26T20:26:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.791+0000 7f9deba1d640 1 -- 172.21.15.37:0/1618968439 shutdown_connections 2024-04-26T20:26:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.791+0000 7f9deba1d640 1 --2- 172.21.15.37:0/1618968439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de406d560 0x7f9de409be30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.791+0000 7f9deba1d640 1 --2- 172.21.15.37:0/1618968439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de409ba50 0x7f9de40b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.791+0000 7f9deba1d640 1 -- 172.21.15.37:0/1618968439 >> 172.21.15.37:0/1618968439 conn(0x7f9de40b4a50 msgr2=0x7f9de40b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.791+0000 7f9deba1d640 1 -- 172.21.15.37:0/1618968439 shutdown_connections 2024-04-26T20:26:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.791+0000 7f9deba1d640 1 -- 172.21.15.37:0/1618968439 wait complete. 2024-04-26T20:26:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.792+0000 7f9deba1d640 1 Processor -- start 2024-04-26T20:26:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.792+0000 7f9deba1d640 1 -- start start 2024-04-26T20:26:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.793+0000 7f9deba1d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de406d560 0x7f9de4160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.793+0000 7f9deba1d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de409ba50 0x7f9de415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.793+0000 7f9deba1d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de415ebd0 0x7f9de41f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.793+0000 7f9deba1d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9de40b4790 con 0x7f9de415ebd0 2024-04-26T20:26:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.793+0000 7f9deba1d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9de40ce190 con 0x7f9de406d560 2024-04-26T20:26:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.793+0000 7f9deba1d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9de415f760 con 0x7f9de409ba50 2024-04-26T20:26:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.793+0000 7f9de9fba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de415ebd0 0x7f9de41f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.793+0000 7f9de9fba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de415ebd0 0x7f9de41f8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40556/0 (socket says 172.21.15.37:40556) 2024-04-26T20:26:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.793+0000 7f9de9fba640 1 -- 172.21.15.37:0/3141713514 learned_addr learned my addr 172.21.15.37:0/3141713514 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.793+0000 7f9de8fb8640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de409ba50 0x7f9de415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.793+0000 7f9de97b9640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de406d560 0x7f9de4160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.794+0000 7f9de9fba640 1 -- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de409ba50 msgr2=0x7f9de415be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.794+0000 7f9de9fba640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de409ba50 0x7f9de415be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.794+0000 7f9de9fba640 1 -- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de406d560 msgr2=0x7f9de4160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:20.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.794+0000 7f9de9fba640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de406d560 0x7f9de4160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.794+0000 7f9de9fba640 1 -- 172.21.15.37:0/3141713514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de41f9390 con 0x7f9de415ebd0 2024-04-26T20:26:20.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.794+0000 7f9de9fba640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de415ebd0 0x7f9de41f8540 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00c370 tx=0x7f9ddc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:20.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.794+0000 7f9de1ffb640 1 -- 172.21.15.37:0/3141713514 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc017070 con 0x7f9de415ebd0 2024-04-26T20:26:20.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.794+0000 7f9de1ffb640 1 -- 172.21.15.37:0/3141713514 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ddc005ce0 con 0x7f9de415ebd0 2024-04-26T20:26:20.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.794+0000 7f9deba1d640 1 -- 172.21.15.37:0/3141713514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de40cd140 con 0x7f9de415ebd0 2024-04-26T20:26:20.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.794+0000 7f9de1ffb640 1 -- 172.21.15.37:0/3141713514 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0072d0 con 0x7f9de415ebd0 2024-04-26T20:26:20.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.794+0000 7f9deba1d640 1 -- 172.21.15.37:0/3141713514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de415fb30 con 0x7f9de415ebd0 2024-04-26T20:26:20.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.795+0000 7f9deba1d640 1 -- 172.21.15.37:0/3141713514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de40693e0 con 0x7f9de415ebd0 2024-04-26T20:26:20.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.795+0000 7f9de1ffb640 1 -- 172.21.15.37:0/3141713514 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f9ddc01a070 con 0x7f9de415ebd0 2024-04-26T20:26:20.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.796+0000 7f9de1ffb640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dd006cbc0 0x7f9dd006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.796+0000 7f9de1ffb640 1 -- 172.21.15.37:0/3141713514 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f9ddc08b240 con 0x7f9de415ebd0 2024-04-26T20:26:20.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.796+0000 7f9de97b9640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dd006cbc0 0x7f9dd006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.796+0000 7f9de97b9640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dd006cbc0 0x7f9dd006f080 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009a60 tx=0x7f9dd4003990 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:20.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.800+0000 7f9de1ffb640 1 -- 172.21.15.37:0/3141713514 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ddc058c90 con 0x7f9de415ebd0 2024-04-26T20:26:21.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.999+0000 7f9deba1d640 1 -- 172.21.15.37:0/3141713514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9de406db40 con 0x7f9de415ebd0 2024-04-26T20:26:21.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:20.999+0000 7f9de1ffb640 1 -- 172.21.15.37:0/3141713514 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7f9ddc05cf90 con 0x7f9de415ebd0 2024-04-26T20:26:21.000 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-26T20:26:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.001+0000 7f9deba1d640 1 -- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dd006cbc0 msgr2=0x7f9dd006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.001+0000 7f9deba1d640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dd006cbc0 0x7f9dd006f080 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009a60 tx=0x7f9dd4003990 comp rx=0 tx=0).stop 2024-04-26T20:26:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.001+0000 7f9deba1d640 1 -- 172.21.15.37:0/3141713514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de415ebd0 msgr2=0x7f9de41f8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.001+0000 7f9deba1d640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de415ebd0 0x7f9de41f8540 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00c370 tx=0x7f9ddc00c830 comp rx=0 tx=0).stop 2024-04-26T20:26:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.001+0000 7f9deba1d640 1 -- 172.21.15.37:0/3141713514 shutdown_connections 2024-04-26T20:26:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.001+0000 7f9deba1d640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de415ebd0 0x7f9de41f8540 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.001+0000 7f9deba1d640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f9dd006cbc0 0x7f9dd006f080 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.001+0000 7f9deba1d640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9de409ba50 0x7f9de415be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.001+0000 7f9deba1d640 1 --2- 172.21.15.37:0/3141713514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de406d560 0x7f9de4160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.001+0000 7f9deba1d640 1 -- 172.21.15.37:0/3141713514 >> 172.21.15.37:0/3141713514 conn(0x7f9de40b4a50 msgr2=0x7f9de40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:21.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.002+0000 7f9deba1d640 1 -- 172.21.15.37:0/3141713514 shutdown_connections 2024-04-26T20:26:21.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.002+0000 7f9deba1d640 1 -- 172.21.15.37:0/3141713514 wait complete. 2024-04-26T20:26:21.014 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T20:26:21.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:21.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T20:26:21.041 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:26:21.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:21.041 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:26:21.097 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:26:21.097 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:26:21.097 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:21.116 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:26:21.116 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:21.175 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.274+0000 7ffa833af640 1 Processor -- start 2024-04-26T20:26:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.274+0000 7ffa833af640 1 -- start start 2024-04-26T20:26:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.275+0000 7ffa833af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c06ea40 0x7ffa7c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.275+0000 7ffa833af640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffa7c0cf830 0x7ffa7c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.275+0000 7ffa833af640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c0b1d70 con 0x7ffa7c162e40 2024-04-26T20:26:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.275+0000 7ffa833af640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7c0d01e0 con 0x7ffa7c0cf830 2024-04-26T20:26:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.275+0000 7ffa833af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7c163220 con 0x7ffa7c06ea40 2024-04-26T20:26:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.275+0000 7ffa8094a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c06ea40 0x7ffa7c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.275+0000 7ffa8194c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffa7c0cf830 0x7ffa7c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.275+0000 7ffa8114b640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ffa7c162e40 0x7ffa7c0b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:48666/0 (socket says 172.21.15.37:48666) 2024-04-26T20:26:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.275+0000 7ffa8094a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c06ea40 0x7ffa7c0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:46218/0 (socket says 172.21.15.37:46218) 2024-04-26T20:26:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.275+0000 7ffa8194c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffa7c0cf830 0x7ffa7c0cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:43590/0 (socket says 172.21.15.37:43590) 2024-04-26T20:26:21.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.275+0000 7ffa8114b640 1 -- 172.21.15.37:0/1766853884 learned_addr learned my addr 172.21.15.37:0/1766853884 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.276+0000 7ffa73fff640 1 -- 172.21.15.37:0/1766853884 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220374209 0 0) 0x7ffa7c0b1d70 con 0x7ffa7c162e40 2024-04-26T20:26:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.276+0000 7ffa73fff640 1 -- 172.21.15.37:0/1766853884 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa68003520 con 0x7ffa7c162e40 2024-04-26T20:26:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.276+0000 7ffa8194c640 1 -- 172.21.15.37:0/1766853884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c06ea40 msgr2=0x7ffa7c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.276+0000 7ffa8194c640 1 --2- 172.21.15.37:0/1766853884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c06ea40 0x7ffa7c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.276+0000 7ffa8194c640 1 -- 172.21.15.37:0/1766853884 >> v1:172.21.15.134:6789/0 conn(0x7ffa7c162e40 legacy=0x7ffa7c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.276+0000 7ffa8194c640 1 -- 172.21.15.37:0/1766853884 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c0d1960 con 0x7ffa7c0cf830 2024-04-26T20:26:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.276+0000 7ffa8094a640 1 --2- 172.21.15.37:0/1766853884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c06ea40 0x7ffa7c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa8194c640 1 --2- 172.21.15.37:0/1766853884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c0cf830 0x7ffa7c0cfca0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009d90 tx=0x7ffa7402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=559b381fac97f7a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa73fff640 1 -- 172.21.15.37:0/1766853884 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74031600 con 0x7ffa7c0cf830 2024-04-26T20:26:21.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa73fff640 1 -- 172.21.15.37:0/1766853884 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa740317a0 con 0x7ffa7c0cf830 2024-04-26T20:26:21.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa73fff640 1 -- 172.21.15.37:0/1766853884 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74031b70 con 0x7ffa7c0cf830 2024-04-26T20:26:21.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa833af640 1 -- 172.21.15.37:0/1766853884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c0cf830 msgr2=0x7ffa7c0cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa833af640 1 --2- 172.21.15.37:0/1766853884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c0cf830 0x7ffa7c0cfca0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009d90 tx=0x7ffa7402a5d0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa833af640 1 -- 172.21.15.37:0/1766853884 shutdown_connections 2024-04-26T20:26:21.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa833af640 1 --2- 172.21.15.37:0/1766853884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c0cf830 0x7ffa7c0cfca0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa833af640 1 --2- 172.21.15.37:0/1766853884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c06ea40 0x7ffa7c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa833af640 1 -- 172.21.15.37:0/1766853884 >> 172.21.15.37:0/1766853884 conn(0x7ffa7c09ba50 msgr2=0x7ffa7c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:21.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa833af640 1 -- 172.21.15.37:0/1766853884 shutdown_connections 2024-04-26T20:26:21.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.277+0000 7ffa833af640 1 -- 172.21.15.37:0/1766853884 wait complete. 2024-04-26T20:26:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa833af640 1 Processor -- start 2024-04-26T20:26:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa833af640 1 -- start start 2024-04-26T20:26:21.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa833af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c06ea40 0x7ffa7c0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa833af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa7c0cf830 0x7ffa7c15c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa833af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c162e40 0x7ffa7c15c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa833af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c0d1f20 con 0x7ffa7c0cf830 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa833af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c0d0eb0 con 0x7ffa7c06ea40 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa833af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7c0ce010 con 0x7ffa7c162e40 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa8094a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa7c0cf830 0x7ffa7c15c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa8094a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa7c0cf830 0x7ffa7c15c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40568/0 (socket says 172.21.15.37:40568) 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa8094a640 1 -- 172.21.15.37:0/4068914921 learned_addr learned my addr 172.21.15.37:0/4068914921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.279+0000 7ffa8114b640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c06ea40 0x7ffa7c0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa8194c640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c162e40 0x7ffa7c15c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa8094a640 1 -- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c162e40 msgr2=0x7ffa7c15c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa8094a640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c162e40 0x7ffa7c15c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa8094a640 1 -- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c06ea40 msgr2=0x7ffa7c0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa8094a640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c06ea40 0x7ffa7c0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa8094a640 1 -- 172.21.15.37:0/4068914921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c162070 con 0x7ffa7c0cf830 2024-04-26T20:26:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa8094a640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa7c0cf830 0x7ffa7c15c450 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00ec10 tx=0x7ffa6c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa71ffb640 1 -- 172.21.15.37:0/4068914921 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c017070 con 0x7ffa7c0cf830 2024-04-26T20:26:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa71ffb640 1 -- 172.21.15.37:0/4068914921 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c00cc30 con 0x7ffa7c0cf830 2024-04-26T20:26:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa833af640 1 -- 172.21.15.37:0/4068914921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c0ca120 con 0x7ffa7c0cf830 2024-04-26T20:26:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa71ffb640 1 -- 172.21.15.37:0/4068914921 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c013470 con 0x7ffa7c0cf830 2024-04-26T20:26:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.280+0000 7ffa833af640 1 -- 172.21.15.37:0/4068914921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c0ca6e0 con 0x7ffa7c0cf830 2024-04-26T20:26:21.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.281+0000 7ffa833af640 1 -- 172.21.15.37:0/4068914921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa7c06a360 con 0x7ffa7c0cf830 2024-04-26T20:26:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.282+0000 7ffa71ffb640 1 -- 172.21.15.37:0/4068914921 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7ffa6c01a070 con 0x7ffa7c0cf830 2024-04-26T20:26:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.282+0000 7ffa71ffb640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa6006cbc0 0x7ffa6006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.282+0000 7ffa71ffb640 1 -- 172.21.15.37:0/4068914921 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7ffa6c08c270 con 0x7ffa7c0cf830 2024-04-26T20:26:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.283+0000 7ffa8114b640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa6006cbc0 0x7ffa6006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.283+0000 7ffa8114b640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa6006cbc0 0x7ffa6006f080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffa78005880 tx=0x7ffa78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:21.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.287+0000 7ffa71ffb640 1 -- 172.21.15.37:0/4068914921 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa6c059cd0 con 0x7ffa7c0cf830 2024-04-26T20:26:21.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.472+0000 7ffa833af640 1 -- 172.21.15.37:0/4068914921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa7c071410 con 0x7ffa7c0cf830 2024-04-26T20:26:21.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.473+0000 7ffa71ffb640 1 -- 172.21.15.37:0/4068914921 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16715 (secure 0 0 0) 0x7ffa6c05dfd0 con 0x7ffa7c0cf830 2024-04-26T20:26:21.474 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:21.474 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":48,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:24:31.476288+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:24:08.047629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:24:08.708738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.f","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:21.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 -- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa6006cbc0 msgr2=0x7ffa6006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa6006cbc0 0x7ffa6006f080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffa78005880 tx=0x7ffa78009040 comp rx=0 tx=0).stop 2024-04-26T20:26:21.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 -- 172.21.15.37:0/4068914921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa7c0cf830 msgr2=0x7ffa7c15c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa7c0cf830 0x7ffa7c15c450 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00ec10 tx=0x7ffa6c00c520 comp rx=0 tx=0).stop 2024-04-26T20:26:21.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 -- 172.21.15.37:0/4068914921 shutdown_connections 2024-04-26T20:26:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffa6006cbc0 0x7ffa6006f080 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c162e40 0x7ffa7c15c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa7c0cf830 0x7ffa7c15c450 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 --2- 172.21.15.37:0/4068914921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c06ea40 0x7ffa7c0b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 -- 172.21.15.37:0/4068914921 >> 172.21.15.37:0/4068914921 conn(0x7ffa7c09ba50 msgr2=0x7ffa7c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 -- 172.21.15.37:0/4068914921 shutdown_connections 2024-04-26T20:26:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.475+0000 7ffa833af640 1 -- 172.21.15.37:0/4068914921 wait complete. 2024-04-26T20:26:21.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:24:08.047629+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:24:08.708738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:26:21.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:26:21.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.590+0000 7ff05ab72640 1 Processor -- start 2024-04-26T20:26:21.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.590+0000 7ff05ab72640 1 -- start start 2024-04-26T20:26:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.591+0000 7ff05ab72640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff054071f70 0x7ff05406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.591+0000 7ff05ab72640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff0540ae1f0 0x7ff0540ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.591+0000 7ff05ab72640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff054072400 con 0x7ff0541609e0 2024-04-26T20:26:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.591+0000 7ff05ab72640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff0540add80 con 0x7ff0540ae1f0 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.591+0000 7ff05ab72640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff054073de0 con 0x7ff054071f70 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.591+0000 7ff05890e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff0541609e0 0x7ff05406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34396/0 (socket says 172.21.15.37:34396) 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.591+0000 7ff05890e640 1 -- 172.21.15.37:0/772758909 learned_addr learned my addr 172.21.15.37:0/772758909 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.591+0000 7ff05910f640 1 --2- 172.21.15.37:0/772758909 >> v2:172.21.15.37:3300/0 conn(0x7ff0540ae1f0 0x7ff0540ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.591+0000 7ff053fff640 1 --2- 172.21.15.37:0/772758909 >> v2:172.21.15.134:3300/0 conn(0x7ff054071f70 0x7ff05406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.591+0000 7ff0537fe640 1 -- 172.21.15.37:0/772758909 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2372393170 0 0) 0x7ff054072400 con 0x7ff0541609e0 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.591+0000 7ff0537fe640 1 -- 172.21.15.37:0/772758909 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff040003520 con 0x7ff0541609e0 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.592+0000 7ff05910f640 1 -- 172.21.15.37:0/772758909 >> v2:172.21.15.134:3300/0 conn(0x7ff054071f70 msgr2=0x7ff05406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.592+0000 7ff05910f640 1 --2- 172.21.15.37:0/772758909 >> v2:172.21.15.134:3300/0 conn(0x7ff054071f70 0x7ff05406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.592+0000 7ff05910f640 1 -- 172.21.15.37:0/772758909 >> v1:172.21.15.37:6789/0 conn(0x7ff0541609e0 legacy=0x7ff05406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.592+0000 7ff05910f640 1 -- 172.21.15.37:0/772758909 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0540740c0 con 0x7ff0540ae1f0 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.592+0000 7ff053fff640 1 --2- 172.21.15.37:0/772758909 >> v2:172.21.15.134:3300/0 conn(0x7ff054071f70 0x7ff05406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.592+0000 7ff05910f640 1 --2- 172.21.15.37:0/772758909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0540ae1f0 0x7ff0540ae660 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff0540ce8c0 tx=0x7ff03c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ee8967c54551e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.592+0000 7ff0537fe640 1 -- 172.21.15.37:0/772758909 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c037070 con 0x7ff0540ae1f0 2024-04-26T20:26:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.592+0000 7ff0537fe640 1 -- 172.21.15.37:0/772758909 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff03c032440 con 0x7ff0540ae1f0 2024-04-26T20:26:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.592+0000 7ff0537fe640 1 -- 172.21.15.37:0/772758909 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c02ae60 con 0x7ff0540ae1f0 2024-04-26T20:26:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.593+0000 7ff05ab72640 1 -- 172.21.15.37:0/772758909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0540ae1f0 msgr2=0x7ff0540ae660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.593+0000 7ff05ab72640 1 --2- 172.21.15.37:0/772758909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0540ae1f0 0x7ff0540ae660 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff0540ce8c0 tx=0x7ff03c02a110 comp rx=0 tx=0).stop 2024-04-26T20:26:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.593+0000 7ff05ab72640 1 -- 172.21.15.37:0/772758909 shutdown_connections 2024-04-26T20:26:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.593+0000 7ff05ab72640 1 --2- 172.21.15.37:0/772758909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0540ae1f0 0x7ff0540ae660 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.593+0000 7ff05ab72640 1 --2- 172.21.15.37:0/772758909 >> v2:172.21.15.134:3300/0 conn(0x7ff054071f70 0x7ff05406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.593+0000 7ff05ab72640 1 -- 172.21.15.37:0/772758909 >> 172.21.15.37:0/772758909 conn(0x7ff0540d0d70 msgr2=0x7ff0540d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.593+0000 7ff05ab72640 1 -- 172.21.15.37:0/772758909 shutdown_connections 2024-04-26T20:26:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.593+0000 7ff05ab72640 1 -- 172.21.15.37:0/772758909 wait complete. 2024-04-26T20:26:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.594+0000 7ff05ab72640 1 Processor -- start 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.594+0000 7ff05ab72640 1 -- start start 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff05ab72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0540ae1f0 0x7ff0540cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff05ab72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0541609e0 0x7ff0540b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff05ab72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0540cffc0 0x7ff0540d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff05ab72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff05406eac0 con 0x7ff0541609e0 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff05ab72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff05406e940 con 0x7ff0540ae1f0 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff05ab72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff0540d1ad0 con 0x7ff0540cffc0 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff053fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0541609e0 0x7ff0540b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff053fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0541609e0 0x7ff0540b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40594/0 (socket says 172.21.15.37:40594) 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff053fff640 1 -- 172.21.15.37:0/2012124426 learned_addr learned my addr 172.21.15.37:0/2012124426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff053fff640 1 -- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0540cffc0 msgr2=0x7ff0540d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff053fff640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0540cffc0 0x7ff0540d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff053fff640 1 -- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0540ae1f0 msgr2=0x7ff0540cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.595+0000 7ff05890e640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0540ae1f0 0x7ff0540cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.596+0000 7ff053fff640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0540ae1f0 0x7ff0540cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.596+0000 7ff053fff640 1 -- 172.21.15.37:0/2012124426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0540b66e0 con 0x7ff0541609e0 2024-04-26T20:26:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.596+0000 7ff05890e640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0540ae1f0 0x7ff0540cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.596+0000 7ff053fff640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0541609e0 0x7ff0540b61a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff044007e70 tx=0x7ff04400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.596+0000 7ff0517fa640 1 -- 172.21.15.37:0/2012124426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04401a070 con 0x7ff0541609e0 2024-04-26T20:26:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.596+0000 7ff0517fa640 1 -- 172.21.15.37:0/2012124426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff044003b90 con 0x7ff0541609e0 2024-04-26T20:26:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.596+0000 7ff05ab72640 1 -- 172.21.15.37:0/2012124426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0540b6980 con 0x7ff0541609e0 2024-04-26T20:26:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.596+0000 7ff0517fa640 1 -- 172.21.15.37:0/2012124426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044004030 con 0x7ff0541609e0 2024-04-26T20:26:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.596+0000 7ff05ab72640 1 -- 172.21.15.37:0/2012124426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0540cf4e0 con 0x7ff0541609e0 2024-04-26T20:26:21.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.597+0000 7ff05ab72640 1 -- 172.21.15.37:0/2012124426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff05406a360 con 0x7ff0541609e0 2024-04-26T20:26:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.597+0000 7ff0517fa640 1 -- 172.21.15.37:0/2012124426 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7ff04401d070 con 0x7ff0541609e0 2024-04-26T20:26:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.598+0000 7ff0517fa640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff03406cbf0 0x7ff03406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.598+0000 7ff0517fa640 1 -- 172.21.15.37:0/2012124426 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7ff04408c2d0 con 0x7ff0541609e0 2024-04-26T20:26:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.598+0000 7ff05890e640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff03406cbf0 0x7ff03406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.598+0000 7ff05890e640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff03406cbf0 0x7ff03406f0b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff04c005880 tx=0x7ff04c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:21.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.602+0000 7ff0517fa640 1 -- 172.21.15.37:0/2012124426 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff044010070 con 0x7ff0541609e0 2024-04-26T20:26:21.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.786+0000 7ff05ab72640 1 -- 172.21.15.37:0/2012124426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0540b4180 con 0x7ff0541609e0 2024-04-26T20:26:21.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.786+0000 7ff0517fa640 1 -- 172.21.15.37:0/2012124426 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7ff04408c0f0 con 0x7ff0541609e0 2024-04-26T20:26:21.787 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:26:21.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 -- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff03406cbf0 msgr2=0x7ff03406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff03406cbf0 0x7ff03406f0b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff04c005880 tx=0x7ff04c009040 comp rx=0 tx=0).stop 2024-04-26T20:26:21.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 -- 172.21.15.37:0/2012124426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0541609e0 msgr2=0x7ff0540b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0541609e0 0x7ff0540b61a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff044007e70 tx=0x7ff04400d4a0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 -- 172.21.15.37:0/2012124426 shutdown_connections 2024-04-26T20:26:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff03406cbf0 0x7ff03406f0b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0540cffc0 0x7ff0540d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0541609e0 0x7ff0540b61a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 --2- 172.21.15.37:0/2012124426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0540ae1f0 0x7ff0540cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 -- 172.21.15.37:0/2012124426 >> 172.21.15.37:0/2012124426 conn(0x7ff0540d0d70 msgr2=0x7ff0540b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 -- 172.21.15.37:0/2012124426 shutdown_connections 2024-04-26T20:26:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.788+0000 7ff05ab72640 1 -- 172.21.15.37:0/2012124426 wait complete. 2024-04-26T20:26:21.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:26:21.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.902+0000 7fe351637640 1 Processor -- start 2024-04-26T20:26:21.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.902+0000 7fe351637640 1 -- start start 2024-04-26T20:26:21.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.903+0000 7fe351637640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe34c0b5510 0x7fe34c0d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.903+0000 7fe351637640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe34c0d3d60 con 0x7fe34c0b6820 2024-04-26T20:26:21.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.903+0000 7fe351637640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe34c0c9cc0 con 0x7fe34c0b58f0 2024-04-26T20:26:21.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.903+0000 7fe351637640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fe34c0b5d80 con 0x7fe34c0b5510 2024-04-26T20:26:21.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.903+0000 7fe34affd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe34c0b5510 0x7fe34c0d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.903+0000 7fe34a7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe34c0b6820 0x7fe34c0d2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34412/0 (socket says 172.21.15.37:34412) 2024-04-26T20:26:21.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.903+0000 7fe34a7fc640 1 -- 172.21.15.37:0/22221106 learned_addr learned my addr 172.21.15.37:0/22221106 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:21.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.904+0000 7fe34affd640 1 -- 172.21.15.37:0/22221106 >> v1:172.21.15.134:6789/0 conn(0x7fe34c0b58f0 legacy=0x7fe34c0d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:21.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.904+0000 7fe34affd640 1 -- 172.21.15.37:0/22221106 >> v1:172.21.15.37:6789/0 conn(0x7fe34c0b6820 legacy=0x7fe34c0d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.904+0000 7fe349ffb640 1 -- 172.21.15.37:0/22221106 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1814476643 0 0) 0x7fe34c0d3d60 con 0x7fe34c0b6820 2024-04-26T20:26:21.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.904+0000 7fe34affd640 1 -- 172.21.15.37:0/22221106 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34c0b5fd0 con 0x7fe34c0b5510 2024-04-26T20:26:21.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.904+0000 7fe34affd640 1 --2- 172.21.15.37:0/22221106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34c0b5510 0x7fe34c0d1890 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0b5f00 tx=0x7fe34402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5309465c3fb10803 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:21.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.905+0000 7fe349ffb640 1 -- 172.21.15.37:0/22221106 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344037070 con 0x7fe34c0b5510 2024-04-26T20:26:21.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.905+0000 7fe349ffb640 1 -- 172.21.15.37:0/22221106 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe34402ae40 con 0x7fe34c0b5510 2024-04-26T20:26:21.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.905+0000 7fe349ffb640 1 -- 172.21.15.37:0/22221106 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344032660 con 0x7fe34c0b5510 2024-04-26T20:26:21.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.905+0000 7fe351637640 1 -- 172.21.15.37:0/22221106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34c0b5510 msgr2=0x7fe34c0d1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.905+0000 7fe351637640 1 --2- 172.21.15.37:0/22221106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34c0b5510 0x7fe34c0d1890 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0b5f00 tx=0x7fe34402a130 comp rx=0 tx=0).stop 2024-04-26T20:26:21.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.905+0000 7fe351637640 1 -- 172.21.15.37:0/22221106 shutdown_connections 2024-04-26T20:26:21.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.905+0000 7fe351637640 1 --2- 172.21.15.37:0/22221106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34c0b5510 0x7fe34c0d1890 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.905+0000 7fe351637640 1 -- 172.21.15.37:0/22221106 >> 172.21.15.37:0/22221106 conn(0x7fe34c0d5160 msgr2=0x7fe34c0d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:21.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.905+0000 7fe351637640 1 -- 172.21.15.37:0/22221106 shutdown_connections 2024-04-26T20:26:21.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.905+0000 7fe351637640 1 -- 172.21.15.37:0/22221106 wait complete. 2024-04-26T20:26:21.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.907+0000 7fe351637640 1 Processor -- start 2024-04-26T20:26:21.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.907+0000 7fe351637640 1 -- start start 2024-04-26T20:26:21.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.907+0000 7fe351637640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c0b5510 0x7fe34c152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.907+0000 7fe351637640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34c0b58f0 0x7fe34c162550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.907+0000 7fe351637640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c0b6820 0x7fe34c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.907+0000 7fe351637640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c0d0880 con 0x7fe34c0b58f0 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.907+0000 7fe351637640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c0d5f60 con 0x7fe34c0b5510 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.907+0000 7fe351637640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe34c0d0b60 con 0x7fe34c0b6820 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.907+0000 7fe34a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34c0b58f0 0x7fe34c162550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.907+0000 7fe34a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34c0b58f0 0x7fe34c162550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40612/0 (socket says 172.21.15.37:40612) 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.907+0000 7fe34a7fc640 1 -- 172.21.15.37:0/2439092179 learned_addr learned my addr 172.21.15.37:0/2439092179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe34affd640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c0b5510 0x7fe34c152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe34b7fe640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c0b6820 0x7fe34c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe34a7fc640 1 -- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c0b6820 msgr2=0x7fe34c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe34a7fc640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c0b6820 0x7fe34c0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe34a7fc640 1 -- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c0b5510 msgr2=0x7fe34c152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe34a7fc640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c0b5510 0x7fe34c152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe34a7fc640 1 -- 172.21.15.37:0/2439092179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34c0b44f0 con 0x7fe34c0b58f0 2024-04-26T20:26:21.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe34b7fe640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c0b6820 0x7fe34c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe34a7fc640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34c0b58f0 0x7fe34c162550 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe34000c460 tx=0x7fe34000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe3337fe640 1 -- 172.21.15.37:0/2439092179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340018070 con 0x7fe34c0b58f0 2024-04-26T20:26:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe3337fe640 1 -- 172.21.15.37:0/2439092179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe340007500 con 0x7fe34c0b58f0 2024-04-26T20:26:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe3337fe640 1 -- 172.21.15.37:0/2439092179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340004a20 con 0x7fe34c0b58f0 2024-04-26T20:26:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe351637640 1 -- 172.21.15.37:0/2439092179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34c0b47d0 con 0x7fe34c0b58f0 2024-04-26T20:26:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.908+0000 7fe351637640 1 -- 172.21.15.37:0/2439092179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe34c0b3230 con 0x7fe34c0b58f0 2024-04-26T20:26:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.909+0000 7fe351637640 1 -- 172.21.15.37:0/2439092179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34c0693e0 con 0x7fe34c0b58f0 2024-04-26T20:26:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.910+0000 7fe3337fe640 1 -- 172.21.15.37:0/2439092179 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fe34001b070 con 0x7fe34c0b58f0 2024-04-26T20:26:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.910+0000 7fe3337fe640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe33806cbc0 0x7fe33806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.910+0000 7fe3337fe640 1 -- 172.21.15.37:0/2439092179 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe34008b2c0 con 0x7fe34c0b58f0 2024-04-26T20:26:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.910+0000 7fe34affd640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe33806cbc0 0x7fe33806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.911+0000 7fe34affd640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe33806cbc0 0x7fe33806f080 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe344034000 tx=0x7fe34402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:21.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:21.914+0000 7fe3337fe640 1 -- 172.21.15.37:0/2439092179 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe340059360 con 0x7fe34c0b58f0 2024-04-26T20:26:22.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.100+0000 7fe351637640 1 -- 172.21.15.37:0/2439092179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe34c0cf260 con 0x7fe34c0b58f0 2024-04-26T20:26:22.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.100+0000 7fe3337fe640 1 -- 172.21.15.37:0/2439092179 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) ==== 104+0+11 (secure 0 0 0) 0x7fe340058c60 con 0x7fe34c0b58f0 2024-04-26T20:26:22.102 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:26:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.102+0000 7fe351637640 1 -- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe33806cbc0 msgr2=0x7fe33806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.102+0000 7fe351637640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe33806cbc0 0x7fe33806f080 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe344034000 tx=0x7fe34402e000 comp rx=0 tx=0).stop 2024-04-26T20:26:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.102+0000 7fe351637640 1 -- 172.21.15.37:0/2439092179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34c0b58f0 msgr2=0x7fe34c162550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.102+0000 7fe351637640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34c0b58f0 0x7fe34c162550 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe34000c460 tx=0x7fe34000c920 comp rx=0 tx=0).stop 2024-04-26T20:26:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.102+0000 7fe351637640 1 -- 172.21.15.37:0/2439092179 shutdown_connections 2024-04-26T20:26:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.102+0000 7fe351637640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe33806cbc0 0x7fe33806f080 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.102+0000 7fe351637640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c0b6820 0x7fe34c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.102+0000 7fe351637640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34c0b58f0 0x7fe34c162550 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.102+0000 7fe351637640 1 --2- 172.21.15.37:0/2439092179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c0b5510 0x7fe34c152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.102+0000 7fe351637640 1 -- 172.21.15.37:0/2439092179 >> 172.21.15.37:0/2439092179 conn(0x7fe34c0d5160 msgr2=0x7fe34c0cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.103+0000 7fe351637640 1 -- 172.21.15.37:0/2439092179 shutdown_connections 2024-04-26T20:26:22.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.103+0000 7fe351637640 1 -- 172.21.15.37:0/2439092179 wait complete. 2024-04-26T20:26:22.115 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:26:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.217+0000 7ff954b1c640 1 Processor -- start 2024-04-26T20:26:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.217+0000 7ff954b1c640 1 -- start start 2024-04-26T20:26:22.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff954b1c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff95015f9c0 0x7ff9500d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff954b1c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff9500cfe00 0x7ff950163a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff954b1c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9500b2550 con 0x7ff9500cf990 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff954b1c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff9500b3760 con 0x7ff9500cfe00 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff954b1c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff95006dcf0 con 0x7ff95015f9c0 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff94ed9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff9500cfe00 0x7ff950163a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff947fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff9500cf990 0x7ff950163380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34422/0 (socket says 172.21.15.37:34422) 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff947fff640 1 -- 172.21.15.37:0/3987427300 learned_addr learned my addr 172.21.15.37:0/3987427300 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff94e59c640 1 --2- 172.21.15.37:0/3987427300 >> v2:172.21.15.134:3300/0 conn(0x7ff95015f9c0 0x7ff9500d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff94ed9d640 1 -- 172.21.15.37:0/3987427300 >> v2:172.21.15.134:3300/0 conn(0x7ff95015f9c0 msgr2=0x7ff9500d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff94ed9d640 1 --2- 172.21.15.37:0/3987427300 >> v2:172.21.15.134:3300/0 conn(0x7ff95015f9c0 0x7ff9500d1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff94ed9d640 1 -- 172.21.15.37:0/3987427300 >> v1:172.21.15.37:6789/0 conn(0x7ff9500cf990 legacy=0x7ff950163380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff94ed9d640 1 -- 172.21.15.37:0/3987427300 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95006dfd0 con 0x7ff9500cfe00 2024-04-26T20:26:22.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.218+0000 7ff94dd9b640 1 -- 172.21.15.37:0/3987427300 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733145772 0 0) 0x7ff938001110 con 0x7ff9500cf990 2024-04-26T20:26:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.219+0000 7ff94ed9d640 1 --2- 172.21.15.37:0/3987427300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9500cfe00 0x7ff950163a90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff9500d2ee0 tx=0x7ff94802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279b7e3a048ad6b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.219+0000 7ff94dd9b640 1 -- 172.21.15.37:0/3987427300 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948007740 con 0x7ff9500cfe00 2024-04-26T20:26:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.219+0000 7ff94dd9b640 1 -- 172.21.15.37:0/3987427300 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9480078e0 con 0x7ff9500cfe00 2024-04-26T20:26:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.219+0000 7ff94dd9b640 1 -- 172.21.15.37:0/3987427300 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948007cb0 con 0x7ff9500cfe00 2024-04-26T20:26:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.220+0000 7ff954b1c640 1 -- 172.21.15.37:0/3987427300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9500cfe00 msgr2=0x7ff950163a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.220+0000 7ff954b1c640 1 --2- 172.21.15.37:0/3987427300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9500cfe00 0x7ff950163a90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff9500d2ee0 tx=0x7ff94802a5f0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.220+0000 7ff954b1c640 1 -- 172.21.15.37:0/3987427300 shutdown_connections 2024-04-26T20:26:22.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.220+0000 7ff954b1c640 1 --2- 172.21.15.37:0/3987427300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9500cfe00 0x7ff950163a90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.220+0000 7ff954b1c640 1 --2- 172.21.15.37:0/3987427300 >> v2:172.21.15.134:3300/0 conn(0x7ff95015f9c0 0x7ff9500d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.220+0000 7ff954b1c640 1 -- 172.21.15.37:0/3987427300 >> 172.21.15.37:0/3987427300 conn(0x7ff9500d2f80 msgr2=0x7ff9500d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:22.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.220+0000 7ff954b1c640 1 -- 172.21.15.37:0/3987427300 shutdown_connections 2024-04-26T20:26:22.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.220+0000 7ff954b1c640 1 -- 172.21.15.37:0/3987427300 wait complete. 2024-04-26T20:26:22.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.221+0000 7ff954b1c640 1 Processor -- start 2024-04-26T20:26:22.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.221+0000 7ff954b1c640 1 -- start start 2024-04-26T20:26:22.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff954b1c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9500cf990 0x7ff9500c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff954b1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9500cfe00 0x7ff950159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff954b1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff95015f9c0 0x7ff950159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff954b1c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff95006e870 con 0x7ff9500cfe00 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff954b1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff95006e590 con 0x7ff95015f9c0 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff954b1c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9500d0270 con 0x7ff9500cf990 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff947fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9500cfe00 0x7ff950159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff947fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9500cfe00 0x7ff950159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40626/0 (socket says 172.21.15.37:40626) 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff947fff640 1 -- 172.21.15.37:0/1553382577 learned_addr learned my addr 172.21.15.37:0/1553382577 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff94e59c640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9500cf990 0x7ff9500c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff94ed9d640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff95015f9c0 0x7ff950159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff947fff640 1 -- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9500cf990 msgr2=0x7ff9500c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff947fff640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9500cf990 0x7ff9500c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff947fff640 1 -- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff95015f9c0 msgr2=0x7ff950159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff947fff640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff95015f9c0 0x7ff950159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff947fff640 1 -- 172.21.15.37:0/1553382577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9500c83a0 con 0x7ff9500cfe00 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff94e59c640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9500cf990 0x7ff9500c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff94ed9d640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff95015f9c0 0x7ff950159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff947fff640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9500cfe00 0x7ff950159210 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff93800c430 tx=0x7ff93800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff946ffd640 1 -- 172.21.15.37:0/1553382577 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380074e0 con 0x7ff9500cfe00 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff946ffd640 1 -- 172.21.15.37:0/1553382577 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9380047e0 con 0x7ff9500cfe00 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff954b1c640 1 -- 172.21.15.37:0/1553382577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9500c7300 con 0x7ff9500cfe00 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.222+0000 7ff946ffd640 1 -- 172.21.15.37:0/1553382577 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938004bd0 con 0x7ff9500cfe00 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.223+0000 7ff954b1c640 1 -- 172.21.15.37:0/1553382577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9500c7ba0 con 0x7ff9500cfe00 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.224+0000 7ff954b1c640 1 -- 172.21.15.37:0/1553382577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9500c76f0 con 0x7ff9500cfe00 2024-04-26T20:26:22.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.224+0000 7ff946ffd640 1 -- 172.21.15.37:0/1553382577 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7ff938011070 con 0x7ff9500cfe00 2024-04-26T20:26:22.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.224+0000 7ff946ffd640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff93006cbc0 0x7ff93006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.225+0000 7ff946ffd640 1 -- 172.21.15.37:0/1553382577 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7ff93808bda0 con 0x7ff9500cfe00 2024-04-26T20:26:22.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.225+0000 7ff94e59c640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff93006cbc0 0x7ff93006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.225+0000 7ff94e59c640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff93006cbc0 0x7ff93006f080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff940008890 tx=0x7ff940008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.229+0000 7ff946ffd640 1 -- 172.21.15.37:0/1553382577 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9500b2550 con 0x7ff9500cfe00 2024-04-26T20:26:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.419+0000 7ff954b1c640 1 -- 172.21.15.37:0/1553382577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff95006a360 con 0x7ff9500cfe00 2024-04-26T20:26:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.419+0000 7ff946ffd640 1 -- 172.21.15.37:0/1553382577 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) ==== 100+0+11 (secure 0 0 0) 0x7ff938059d60 con 0x7ff9500cfe00 2024-04-26T20:26:22.420 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 -- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff93006cbc0 msgr2=0x7ff93006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff93006cbc0 0x7ff93006f080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff940008890 tx=0x7ff940008040 comp rx=0 tx=0).stop 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 -- 172.21.15.37:0/1553382577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9500cfe00 msgr2=0x7ff950159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9500cfe00 0x7ff950159210 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff93800c430 tx=0x7ff93800c8f0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 -- 172.21.15.37:0/1553382577 shutdown_connections 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff95015f9c0 0x7ff950159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9500cfe00 0x7ff950159210 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff93006cbc0 0x7ff93006f080 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 --2- 172.21.15.37:0/1553382577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9500cf990 0x7ff9500c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 -- 172.21.15.37:0/1553382577 >> 172.21.15.37:0/1553382577 conn(0x7ff9500d2f80 msgr2=0x7ff950075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 -- 172.21.15.37:0/1553382577 shutdown_connections 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.421+0000 7ff954b1c640 1 -- 172.21.15.37:0/1553382577 wait complete. 2024-04-26T20:26:22.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:26:22.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.534+0000 7f92db2bd640 1 Processor -- start 2024-04-26T20:26:22.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.534+0000 7f92db2bd640 1 -- start start 2024-04-26T20:26:22.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.535+0000 7f92db2bd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f92d4162e40 0x7f92d40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.535+0000 7f92db2bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d40b2050 0x7f92d40cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.535+0000 7f92db2bd640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d40b1d70 con 0x7f92d40cfd70 2024-04-26T20:26:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.535+0000 7f92db2bd640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f92d4163220 con 0x7f92d4162e40 2024-04-26T20:26:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.535+0000 7f92db2bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f92d40d0eb0 con 0x7f92d40b2050 2024-04-26T20:26:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.535+0000 7f92d9059640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f92d4162e40 0x7f92d40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.535+0000 7f92d8858640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d40b2050 0x7f92d40cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.535+0000 7f92d985a640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f92d40cfd70 0x7f92d40ce010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.37:48684/0 (socket says 172.21.15.37:48684) 2024-04-26T20:26:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.535+0000 7f92d985a640 1 -- 172.21.15.37:0/669754428 learned_addr learned my addr 172.21.15.37:0/669754428 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.535+0000 7f92d9059640 1 -- 172.21.15.37:0/669754428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d40b2050 msgr2=0x7f92d40cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.536+0000 7f92d9059640 1 --2- 172.21.15.37:0/669754428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d40b2050 0x7f92d40cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.536+0000 7f92d9059640 1 -- 172.21.15.37:0/669754428 >> v1:172.21.15.134:6789/0 conn(0x7f92d40cfd70 legacy=0x7f92d40ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.536+0000 7f92cbfff640 1 -- 172.21.15.37:0/669754428 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1324975527 0 0) 0x7f92d40b1d70 con 0x7f92d40cfd70 2024-04-26T20:26:22.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.536+0000 7f92d9059640 1 -- 172.21.15.37:0/669754428 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d40d1960 con 0x7f92d4162e40 2024-04-26T20:26:22.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.536+0000 7f92d8858640 1 --2- 172.21.15.37:0/669754428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d40b2050 0x7f92d40cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:22.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.536+0000 7f92d9059640 1 --2- 172.21.15.37:0/669754428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d4162e40 0x7f92d40d1340 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92d00097a0 tx=0x7f92d002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7269815c0fb33632 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.536+0000 7f92cbfff640 1 -- 172.21.15.37:0/669754428 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0037070 con 0x7f92d4162e40 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.536+0000 7f92cbfff640 1 -- 172.21.15.37:0/669754428 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d002ae50 con 0x7f92d4162e40 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.537+0000 7f92cbfff640 1 -- 172.21.15.37:0/669754428 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0032660 con 0x7f92d4162e40 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.537+0000 7f92db2bd640 1 -- 172.21.15.37:0/669754428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d4162e40 msgr2=0x7f92d40d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.537+0000 7f92db2bd640 1 --2- 172.21.15.37:0/669754428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d4162e40 0x7f92d40d1340 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92d00097a0 tx=0x7f92d002a100 comp rx=0 tx=0).stop 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.537+0000 7f92db2bd640 1 -- 172.21.15.37:0/669754428 shutdown_connections 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.537+0000 7f92db2bd640 1 --2- 172.21.15.37:0/669754428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d40b2050 0x7f92d40cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.537+0000 7f92db2bd640 1 --2- 172.21.15.37:0/669754428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d4162e40 0x7f92d40d1340 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.537+0000 7f92db2bd640 1 -- 172.21.15.37:0/669754428 >> 172.21.15.37:0/669754428 conn(0x7f92d409ba50 msgr2=0x7f92d409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.537+0000 7f92db2bd640 1 -- 172.21.15.37:0/669754428 shutdown_connections 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.537+0000 7f92db2bd640 1 -- 172.21.15.37:0/669754428 wait complete. 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.538+0000 7f92db2bd640 1 Processor -- start 2024-04-26T20:26:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.538+0000 7f92db2bd640 1 -- start start 2024-04-26T20:26:22.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92db2bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d40b2050 0x7f92d415e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92db2bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d40cfd70 0x7f92d406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92db2bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d4162e40 0x7f92d4161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92db2bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92d40d1f20 con 0x7f92d4162e40 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92db2bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92d40d01a0 con 0x7f92d40b2050 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92db2bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f92d40b24b0 con 0x7f92d40cfd70 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92d985a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d4162e40 0x7f92d4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92d985a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d4162e40 0x7f92d4161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40642/0 (socket says 172.21.15.37:40642) 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92d985a640 1 -- 172.21.15.37:0/3350165308 learned_addr learned my addr 172.21.15.37:0/3350165308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92d8858640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d40cfd70 0x7f92d406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92d9059640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d40b2050 0x7f92d415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92d985a640 1 -- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d40cfd70 msgr2=0x7f92d406e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92d985a640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d40cfd70 0x7f92d406e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92d985a640 1 -- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d40b2050 msgr2=0x7f92d415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.539+0000 7f92d985a640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d40b2050 0x7f92d415e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.540+0000 7f92d985a640 1 -- 172.21.15.37:0/3350165308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d415fdc0 con 0x7f92d4162e40 2024-04-26T20:26:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.540+0000 7f92d985a640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d4162e40 0x7f92d4161680 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00c7d0 tx=0x7f92cc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.540+0000 7f92c9ffb640 1 -- 172.21.15.37:0/3350165308 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc018070 con 0x7f92d4162e40 2024-04-26T20:26:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.540+0000 7f92c9ffb640 1 -- 172.21.15.37:0/3350165308 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92cc00a640 con 0x7f92d4162e40 2024-04-26T20:26:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.540+0000 7f92db2bd640 1 -- 172.21.15.37:0/3350165308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d41600a0 con 0x7f92d4162e40 2024-04-26T20:26:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.540+0000 7f92c9ffb640 1 -- 172.21.15.37:0/3350165308 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc00aa70 con 0x7f92d4162e40 2024-04-26T20:26:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.540+0000 7f92db2bd640 1 -- 172.21.15.37:0/3350165308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d4160660 con 0x7f92d4162e40 2024-04-26T20:26:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.541+0000 7f92db2bd640 1 -- 172.21.15.37:0/3350165308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d406a360 con 0x7f92d4162e40 2024-04-26T20:26:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.541+0000 7f92c9ffb640 1 -- 172.21.15.37:0/3350165308 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f92cc01b070 con 0x7f92d4162e40 2024-04-26T20:26:22.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.542+0000 7f92c9ffb640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b806cbc0 0x7f92b806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.542+0000 7f92c9ffb640 1 -- 172.21.15.37:0/3350165308 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f92cc08b570 con 0x7f92d4162e40 2024-04-26T20:26:22.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.542+0000 7f92d9059640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b806cbc0 0x7f92b806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.543+0000 7f92d9059640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b806cbc0 0x7f92b806f080 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f92d0034000 tx=0x7f92d002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.546+0000 7f92c9ffb640 1 -- 172.21.15.37:0/3350165308 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92cc058fc0 con 0x7f92d4162e40 2024-04-26T20:26:22.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.746+0000 7f92db2bd640 1 -- 172.21.15.37:0/3350165308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f92d40c9b40 con 0x7f92d4162e40 2024-04-26T20:26:22.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.746+0000 7f92c9ffb640 1 -- 172.21.15.37:0/3350165308 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7f92cc05d2c0 con 0x7f92d4162e40 2024-04-26T20:26:22.747 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-26T20:26:22.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 -- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b806cbc0 msgr2=0x7f92b806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b806cbc0 0x7f92b806f080 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f92d0034000 tx=0x7f92d002e000 comp rx=0 tx=0).stop 2024-04-26T20:26:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 -- 172.21.15.37:0/3350165308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d4162e40 msgr2=0x7f92d4161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d4162e40 0x7f92d4161680 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00c7d0 tx=0x7f92cc00cc90 comp rx=0 tx=0).stop 2024-04-26T20:26:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 -- 172.21.15.37:0/3350165308 shutdown_connections 2024-04-26T20:26:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d4162e40 0x7f92d4161680 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92b806cbc0 0x7f92b806f080 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d40cfd70 0x7f92d406e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 --2- 172.21.15.37:0/3350165308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d40b2050 0x7f92d415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 -- 172.21.15.37:0/3350165308 >> 172.21.15.37:0/3350165308 conn(0x7f92d409ba50 msgr2=0x7f92d40705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 -- 172.21.15.37:0/3350165308 shutdown_connections 2024-04-26T20:26:22.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:22.748+0000 7f92db2bd640 1 -- 172.21.15.37:0/3350165308 wait complete. 2024-04-26T20:26:22.760 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T20:26:22.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:22.760 DEBUG:teuthology.orchestra.run.smithi134:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T20:26:22.792 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:26:22.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:22.792 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T20:26:22.850 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:26:22.850 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:26:22.850 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:22.866 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=371 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:26:22.866 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:26:22.866 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:26:22.866 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:22.907 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17743 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:26:22.908 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:23.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.006+0000 7f06a87cf640 1 Processor -- start 2024-04-26T20:26:23.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.007+0000 7f06a87cf640 1 -- start start 2024-04-26T20:26:23.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.007+0000 7f06a87cf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f06a006ea40 0x7f06a00d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.007+0000 7f06a87cf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a00b1d70 con 0x7f06a0162e40 2024-04-26T20:26:23.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.007+0000 7f06a87cf640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a00d1960 con 0x7f06a00cf830 2024-04-26T20:26:23.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.007+0000 7f06a87cf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f06a0163220 con 0x7f06a006ea40 2024-04-26T20:26:23.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.008+0000 7f06a656b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f06a0162e40 0x7f06a00b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34424/0 (socket says 172.21.15.37:34424) 2024-04-26T20:26:23.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.008+0000 7f06a656b640 1 -- 172.21.15.37:0/1564782935 learned_addr learned my addr 172.21.15.37:0/1564782935 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:23.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.008+0000 7f06a5d6a640 1 --2- 172.21.15.37:0/1564782935 >> v2:172.21.15.37:3300/0 conn(0x7f06a006ea40 0x7f06a00d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.008+0000 7f06a5569640 1 -- 172.21.15.37:0/1564782935 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654112929 0 0) 0x7f06a00b1d70 con 0x7f06a0162e40 2024-04-26T20:26:23.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.008+0000 7f06a5569640 1 -- 172.21.15.37:0/1564782935 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0688003520 con 0x7f06a0162e40 2024-04-26T20:26:23.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.008+0000 7f06a5d6a640 1 -- 172.21.15.37:0/1564782935 >> v1:172.21.15.134:6789/0 conn(0x7f06a00cf830 legacy=0x7f06a00cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:23.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.008+0000 7f06a5d6a640 1 -- 172.21.15.37:0/1564782935 >> v1:172.21.15.37:6789/0 conn(0x7f06a0162e40 legacy=0x7f06a00b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.008+0000 7f06a5d6a640 1 -- 172.21.15.37:0/1564782935 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a00d1c40 con 0x7f06a006ea40 2024-04-26T20:26:23.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.009+0000 7f06a5d6a640 1 --2- 172.21.15.37:0/1564782935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a006ea40 0x7f06a00d1340 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f06940097a0 tx=0x7f069402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca76ed19e1b837af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.009+0000 7f06a5569640 1 -- 172.21.15.37:0/1564782935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694037070 con 0x7f06a006ea40 2024-04-26T20:26:23.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.009+0000 7f06a5569640 1 -- 172.21.15.37:0/1564782935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069402ae50 con 0x7f06a006ea40 2024-04-26T20:26:23.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.009+0000 7f06a5569640 1 -- 172.21.15.37:0/1564782935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694032660 con 0x7f06a006ea40 2024-04-26T20:26:23.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.009+0000 7f06a87cf640 1 -- 172.21.15.37:0/1564782935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a006ea40 msgr2=0x7f06a00d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.009+0000 7f06a87cf640 1 --2- 172.21.15.37:0/1564782935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a006ea40 0x7f06a00d1340 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f06940097a0 tx=0x7f069402a140 comp rx=0 tx=0).stop 2024-04-26T20:26:23.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.009+0000 7f06a87cf640 1 -- 172.21.15.37:0/1564782935 shutdown_connections 2024-04-26T20:26:23.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.009+0000 7f06a87cf640 1 --2- 172.21.15.37:0/1564782935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a006ea40 0x7f06a00d1340 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.009+0000 7f06a87cf640 1 -- 172.21.15.37:0/1564782935 >> 172.21.15.37:0/1564782935 conn(0x7f06a009ba50 msgr2=0x7f06a009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:23.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.009+0000 7f06a87cf640 1 -- 172.21.15.37:0/1564782935 shutdown_connections 2024-04-26T20:26:23.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.009+0000 7f06a87cf640 1 -- 172.21.15.37:0/1564782935 wait complete. 2024-04-26T20:26:23.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.011+0000 7f06a87cf640 1 Processor -- start 2024-04-26T20:26:23.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.011+0000 7f06a87cf640 1 -- start start 2024-04-26T20:26:23.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.011+0000 7f06a87cf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a006ea40 0x7f06a00c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.011+0000 7f06a87cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a00cf830 0x7f06a0153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.011+0000 7f06a87cf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0162e40 0x7f06a015ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.011+0000 7f06a87cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06a00ce010 con 0x7f06a00cf830 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.011+0000 7f06a87cf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06a00d0eb0 con 0x7f06a0162e40 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.011+0000 7f06a87cf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f06a00ce190 con 0x7f06a006ea40 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.011+0000 7f06a5d6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a00cf830 0x7f06a0153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.011+0000 7f06a5d6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a00cf830 0x7f06a0153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40668/0 (socket says 172.21.15.37:40668) 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.011+0000 7f06a5d6a640 1 -- 172.21.15.37:0/402632202 learned_addr learned my addr 172.21.15.37:0/402632202 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f06a6d6c640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0162e40 0x7f06a015ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f06a656b640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a006ea40 0x7f06a00c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f06a5d6a640 1 -- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a006ea40 msgr2=0x7f06a00c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f06a5d6a640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a006ea40 0x7f06a00c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f06a5d6a640 1 -- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0162e40 msgr2=0x7f06a015ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f06a5d6a640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0162e40 0x7f06a015ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f06a5d6a640 1 -- 172.21.15.37:0/402632202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a00c8390 con 0x7f06a00cf830 2024-04-26T20:26:23.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f06a656b640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a006ea40 0x7f06a00c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:23.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f06a5d6a640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a00cf830 0x7f06a0153b70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0694000c00 tx=0x7f06940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f069effd640 1 -- 172.21.15.37:0/402632202 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694041070 con 0x7f06a00cf830 2024-04-26T20:26:23.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f069effd640 1 -- 172.21.15.37:0/402632202 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069403c460 con 0x7f06a00cf830 2024-04-26T20:26:23.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f06a87cf640 1 -- 172.21.15.37:0/402632202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a00c8670 con 0x7f06a00cf830 2024-04-26T20:26:23.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f069effd640 1 -- 172.21.15.37:0/402632202 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694037040 con 0x7f06a00cf830 2024-04-26T20:26:23.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.012+0000 7f06a87cf640 1 -- 172.21.15.37:0/402632202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a00c8c30 con 0x7f06a00cf830 2024-04-26T20:26:23.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.013+0000 7f06a87cf640 1 -- 172.21.15.37:0/402632202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a006a360 con 0x7f06a00cf830 2024-04-26T20:26:23.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.014+0000 7f069effd640 1 -- 172.21.15.37:0/402632202 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f0694044070 con 0x7f06a00cf830 2024-04-26T20:26:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.014+0000 7f069effd640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f068c06cbc0 0x7f068c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.014+0000 7f069effd640 1 -- 172.21.15.37:0/402632202 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f06940ac9c0 con 0x7f06a00cf830 2024-04-26T20:26:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.014+0000 7f06a656b640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f068c06cbc0 0x7f068c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.015+0000 7f06a656b640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f068c06cbc0 0x7f068c06f080 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0690005c00 tx=0x7f0690009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.019+0000 7f069effd640 1 -- 172.21.15.37:0/402632202 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f069407a970 con 0x7f06a00cf830 2024-04-26T20:26:23.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.242+0000 7f06a87cf640 1 -- 172.21.15.37:0/402632202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06a00c8040 con 0x7f06a00cf830 2024-04-26T20:26:23.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.243+0000 7f069effd640 1 -- 172.21.15.37:0/402632202 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5178 (secure 0 0 0) 0x7f069407a270 con 0x7f06a00cf830 2024-04-26T20:26:23.244 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:23.244 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/689917778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":689917778},{"type":"v1","addr":"172.21.15.134:6836","nonce":689917778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5362,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/497972484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":497972484},{"type":"v1","addr":"172.21.15.134:6837","nonce":497972484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/221502372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":221502372},{"type":"v1","addr":"172.21.15.37:6833","nonce":221502372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:24:09.772695+0000","modified":"2024-04-26T20:24:11.765632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1915220099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1915220099},{"type":"v1","addr":"172.21.15.37:6835","nonce":1915220099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5688,"qdb_cluster":[5688]},"id":3}]} 2024-04-26T20:26:23.244 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.244+0000 7f06a87cf640 1 -- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f068c06cbc0 msgr2=0x7f068c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.244+0000 7f06a87cf640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f068c06cbc0 0x7f068c06f080 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0690005c00 tx=0x7f0690009040 comp rx=0 tx=0).stop 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.244+0000 7f06a87cf640 1 -- 172.21.15.37:0/402632202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a00cf830 msgr2=0x7f06a0153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.244+0000 7f06a87cf640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a00cf830 0x7f06a0153b70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0694000c00 tx=0x7f06940046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.244+0000 7f06a87cf640 1 -- 172.21.15.37:0/402632202 shutdown_connections 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.244+0000 7f06a87cf640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0162e40 0x7f06a015ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.244+0000 7f06a87cf640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06a00cf830 0x7f06a0153b70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.244+0000 7f06a87cf640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f068c06cbc0 0x7f068c06f080 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.244+0000 7f06a87cf640 1 --2- 172.21.15.37:0/402632202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a006ea40 0x7f06a00c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.244+0000 7f06a87cf640 1 -- 172.21.15.37:0/402632202 >> 172.21.15.37:0/402632202 conn(0x7f06a009ba50 msgr2=0x7f06a00cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.244+0000 7f06a87cf640 1 -- 172.21.15.37:0/402632202 shutdown_connections 2024-04-26T20:26:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.245+0000 7f06a87cf640 1 -- 172.21.15.37:0/402632202 wait complete. 2024-04-26T20:26:23.257 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.356+0000 7fda6699e640 1 Processor -- start 2024-04-26T20:26:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.356+0000 7fda6699e640 1 -- start start 2024-04-26T20:26:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.357+0000 7fda6699e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda600d0f90 0x7fda600d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.357+0000 7fda6699e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fda6006e360 0x7fda60168d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.357+0000 7fda6699e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda6015f5a0 con 0x7fda6015f790 2024-04-26T20:26:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.357+0000 7fda6699e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fda601605f0 con 0x7fda6006e360 2024-04-26T20:26:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.357+0000 7fda6699e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fda600d2080 con 0x7fda600d0f90 2024-04-26T20:26:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.357+0000 7fda6599c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fda6015f790 0x7fda6015fb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34438/0 (socket says 172.21.15.37:34438) 2024-04-26T20:26:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.357+0000 7fda6599c640 1 -- 172.21.15.37:0/1589795195 learned_addr learned my addr 172.21.15.37:0/1589795195 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.357+0000 7fda6619d640 1 --2- 172.21.15.37:0/1589795195 >> v2:172.21.15.37:3300/0 conn(0x7fda6006e360 0x7fda60168d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.358+0000 7fda6519b640 1 --2- 172.21.15.37:0/1589795195 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda600d0f90 0x7fda600d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.358+0000 7fda6499a640 1 -- 172.21.15.37:0/1589795195 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1160828691 0 0) 0x7fda6015f5a0 con 0x7fda6015f790 2024-04-26T20:26:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.358+0000 7fda6499a640 1 -- 172.21.15.37:0/1589795195 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda4c003520 con 0x7fda6015f790 2024-04-26T20:26:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.358+0000 7fda6619d640 1 -- 172.21.15.37:0/1589795195 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda600d0f90 msgr2=0x7fda600d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.358+0000 7fda6619d640 1 --2- 172.21.15.37:0/1589795195 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda600d0f90 0x7fda600d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.358+0000 7fda6619d640 1 -- 172.21.15.37:0/1589795195 >> v1:172.21.15.37:6789/0 conn(0x7fda6015f790 legacy=0x7fda6015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.358+0000 7fda6619d640 1 -- 172.21.15.37:0/1589795195 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda600d1470 con 0x7fda6006e360 2024-04-26T20:26:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.358+0000 7fda6619d640 1 --2- 172.21.15.37:0/1589795195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda6006e360 0x7fda60168d00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009d90 tx=0x7fda5c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4a1357354fcce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.359+0000 7fda6499a640 1 -- 172.21.15.37:0/1589795195 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c031600 con 0x7fda6006e360 2024-04-26T20:26:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.359+0000 7fda6499a640 1 -- 172.21.15.37:0/1589795195 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda5c0317a0 con 0x7fda6006e360 2024-04-26T20:26:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.359+0000 7fda6499a640 1 -- 172.21.15.37:0/1589795195 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c031b70 con 0x7fda6006e360 2024-04-26T20:26:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.359+0000 7fda6699e640 1 -- 172.21.15.37:0/1589795195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda6006e360 msgr2=0x7fda60168d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.359+0000 7fda6699e640 1 --2- 172.21.15.37:0/1589795195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda6006e360 0x7fda60168d00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009d90 tx=0x7fda5c02a5d0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.359+0000 7fda6699e640 1 -- 172.21.15.37:0/1589795195 shutdown_connections 2024-04-26T20:26:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.359+0000 7fda6699e640 1 --2- 172.21.15.37:0/1589795195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda6006e360 0x7fda60168d00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.359+0000 7fda6699e640 1 --2- 172.21.15.37:0/1589795195 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda600d0f90 0x7fda600d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.359+0000 7fda6699e640 1 -- 172.21.15.37:0/1589795195 >> 172.21.15.37:0/1589795195 conn(0x7fda600d2d90 msgr2=0x7fda600d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.359+0000 7fda6699e640 1 -- 172.21.15.37:0/1589795195 shutdown_connections 2024-04-26T20:26:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.359+0000 7fda6699e640 1 -- 172.21.15.37:0/1589795195 wait complete. 2024-04-26T20:26:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.361+0000 7fda6699e640 1 Processor -- start 2024-04-26T20:26:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.361+0000 7fda6699e640 1 -- start start 2024-04-26T20:26:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.361+0000 7fda6699e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda6006e360 0x7fda60153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.361+0000 7fda6699e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda600d0f90 0x7fda60158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.361+0000 7fda6699e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda6015f790 0x7fda60154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.361+0000 7fda6699e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fda600d1d10 con 0x7fda6015f790 2024-04-26T20:26:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.361+0000 7fda6699e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda600d1a30 con 0x7fda600d0f90 2024-04-26T20:26:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.361+0000 7fda6699e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fda600d0560 con 0x7fda6006e360 2024-04-26T20:26:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.361+0000 7fda6619d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda6015f790 0x7fda60154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6619d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda6015f790 0x7fda60154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40688/0 (socket says 172.21.15.37:40688) 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6619d640 1 -- 172.21.15.37:0/2690482251 learned_addr learned my addr 172.21.15.37:0/2690482251 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6599c640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda6006e360 0x7fda60153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6519b640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda600d0f90 0x7fda60158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6619d640 1 -- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda6006e360 msgr2=0x7fda60153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6619d640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda6006e360 0x7fda60153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6619d640 1 -- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda600d0f90 msgr2=0x7fda60158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6619d640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda600d0f90 0x7fda60158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6619d640 1 -- 172.21.15.37:0/2690482251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda60157770 con 0x7fda6015f790 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6519b640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda600d0f90 0x7fda60158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6619d640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda6015f790 0x7fda60154050 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fda5c02abc0 tx=0x7fda5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda567fc640 1 -- 172.21.15.37:0/2690482251 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c040070 con 0x7fda6015f790 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda567fc640 1 -- 172.21.15.37:0/2690482251 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda5c030b60 con 0x7fda6015f790 2024-04-26T20:26:23.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6699e640 1 -- 172.21.15.37:0/2690482251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda60157150 con 0x7fda6015f790 2024-04-26T20:26:23.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda567fc640 1 -- 172.21.15.37:0/2690482251 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c009450 con 0x7fda6015f790 2024-04-26T20:26:23.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.362+0000 7fda6699e640 1 -- 172.21.15.37:0/2690482251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda60157cf0 con 0x7fda6015f790 2024-04-26T20:26:23.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.363+0000 7fda6699e640 1 -- 172.21.15.37:0/2690482251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda60076890 con 0x7fda6015f790 2024-04-26T20:26:23.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.364+0000 7fda567fc640 1 -- 172.21.15.37:0/2690482251 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fda5c039040 con 0x7fda6015f790 2024-04-26T20:26:23.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.364+0000 7fda567fc640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fda4406cbc0 0x7fda4406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.364+0000 7fda567fc640 1 -- 172.21.15.37:0/2690482251 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fda5c0ad6d0 con 0x7fda6015f790 2024-04-26T20:26:23.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.365+0000 7fda6599c640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fda4406cbc0 0x7fda4406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.365+0000 7fda6599c640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fda4406cbc0 0x7fda4406f080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fda58005880 tx=0x7fda58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.369+0000 7fda567fc640 1 -- 172.21.15.37:0/2690482251 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda5c07b6b0 con 0x7fda6015f790 2024-04-26T20:26:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.558+0000 7fda6699e640 1 -- 172.21.15.37:0/2690482251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda60070bb0 con 0x7fda6015f790 2024-04-26T20:26:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.559+0000 7fda567fc640 1 -- 172.21.15.37:0/2690482251 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16715 (secure 0 0 0) 0x7fda5c07afb0 con 0x7fda6015f790 2024-04-26T20:26:23.560 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:23.560 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":48,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:24:31.476288+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:24:08.047629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:24:08.708738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.f","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 -- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fda4406cbc0 msgr2=0x7fda4406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fda4406cbc0 0x7fda4406f080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fda58005880 tx=0x7fda58009040 comp rx=0 tx=0).stop 2024-04-26T20:26:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 -- 172.21.15.37:0/2690482251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda6015f790 msgr2=0x7fda60154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda6015f790 0x7fda60154050 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fda5c02abc0 tx=0x7fda5c004a40 comp rx=0 tx=0).stop 2024-04-26T20:26:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 -- 172.21.15.37:0/2690482251 shutdown_connections 2024-04-26T20:26:23.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda6015f790 0x7fda60154050 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda600d0f90 0x7fda60158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fda4406cbc0 0x7fda4406f080 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 --2- 172.21.15.37:0/2690482251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda6006e360 0x7fda60153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 -- 172.21.15.37:0/2690482251 >> 172.21.15.37:0/2690482251 conn(0x7fda600d2d90 msgr2=0x7fda600704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:23.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 -- 172.21.15.37:0/2690482251 shutdown_connections 2024-04-26T20:26:23.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.561+0000 7fda6699e640 1 -- 172.21.15.37:0/2690482251 wait complete. 2024-04-26T20:26:23.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:21:37.350254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:24:08.047629+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:24:08.708738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:26:23.574 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:26:23.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.677+0000 7eff81d11640 1 Processor -- start 2024-04-26T20:26:23.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.677+0000 7eff81d11640 1 -- start start 2024-04-26T20:26:23.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.677+0000 7eff81d11640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7eff7c15d320 0x7eff7c0d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.677+0000 7eff81d11640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7eff7c0d1090 0x7eff7c0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.677+0000 7eff81d11640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff7c15d700 con 0x7eff7c0cf990 2024-04-26T20:26:23.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.677+0000 7eff81d11640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7eff7c0ceeb0 con 0x7eff7c15d320 2024-04-26T20:26:23.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.677+0000 7eff81d11640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7eff7c0ce8f0 con 0x7eff7c0d1090 2024-04-26T20:26:23.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.678+0000 7eff7b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7eff7c15d320 0x7eff7c0d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.678+0000 7eff7bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7eff7c0cf990 0x7eff7c0cfe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34440/0 (socket says 172.21.15.37:34440) 2024-04-26T20:26:23.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.678+0000 7eff7b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7eff7c15d320 0x7eff7c0d0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40702/0 (socket says 172.21.15.37:40702) 2024-04-26T20:26:23.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.678+0000 7eff7bfff640 1 -- 172.21.15.37:0/1097828251 learned_addr learned my addr 172.21.15.37:0/1097828251 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:23.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.678+0000 7eff7affd640 1 --2- 172.21.15.37:0/1097828251 >> v2:172.21.15.134:3300/0 conn(0x7eff7c0d1090 0x7eff7c0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.678+0000 7eff7b7fe640 1 -- 172.21.15.37:0/1097828251 >> v2:172.21.15.134:3300/0 conn(0x7eff7c0d1090 msgr2=0x7eff7c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:23.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.678+0000 7eff7b7fe640 1 --2- 172.21.15.37:0/1097828251 >> v2:172.21.15.134:3300/0 conn(0x7eff7c0d1090 0x7eff7c0d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.678+0000 7eff7b7fe640 1 -- 172.21.15.37:0/1097828251 >> v1:172.21.15.37:6789/0 conn(0x7eff7c0cf990 legacy=0x7eff7c0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.678+0000 7eff7b7fe640 1 -- 172.21.15.37:0/1097828251 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7c0cebd0 con 0x7eff7c15d320 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff7b7fe640 1 --2- 172.21.15.37:0/1097828251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7c15d320 0x7eff7c0d0b50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0097a0 tx=0x7eff6c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2bbd379c7d036 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff7a7fc640 1 -- 172.21.15.37:0/1097828251 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c037070 con 0x7eff7c15d320 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff7a7fc640 1 -- 172.21.15.37:0/1097828251 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff6c02ae50 con 0x7eff7c15d320 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff7a7fc640 1 -- 172.21.15.37:0/1097828251 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c032660 con 0x7eff7c15d320 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff81d11640 1 -- 172.21.15.37:0/1097828251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7c15d320 msgr2=0x7eff7c0d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff81d11640 1 --2- 172.21.15.37:0/1097828251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7c15d320 0x7eff7c0d0b50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0097a0 tx=0x7eff6c02a140 comp rx=0 tx=0).stop 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff81d11640 1 -- 172.21.15.37:0/1097828251 shutdown_connections 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff81d11640 1 --2- 172.21.15.37:0/1097828251 >> v2:172.21.15.134:3300/0 conn(0x7eff7c0d1090 0x7eff7c0d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff81d11640 1 --2- 172.21.15.37:0/1097828251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7c15d320 0x7eff7c0d0b50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff81d11640 1 -- 172.21.15.37:0/1097828251 >> 172.21.15.37:0/1097828251 conn(0x7eff7c0d2f80 msgr2=0x7eff7c0d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff81d11640 1 -- 172.21.15.37:0/1097828251 shutdown_connections 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.679+0000 7eff81d11640 1 -- 172.21.15.37:0/1097828251 wait complete. 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.680+0000 7eff81d11640 1 Processor -- start 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.681+0000 7eff81d11640 1 -- start start 2024-04-26T20:26:23.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.681+0000 7eff81d11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c0cf990 0x7eff7c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.681+0000 7eff81d11640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7c0d1090 0x7eff7c0c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.681+0000 7eff81d11640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff7c15d320 0x7eff7c153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.681+0000 7eff81d11640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c0ce550 con 0x7eff7c0d1090 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.681+0000 7eff81d11640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c0b2cd0 con 0x7eff7c0cf990 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.681+0000 7eff81d11640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7eff7c0ce6d0 con 0x7eff7c15d320 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff7affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7c0d1090 0x7eff7c0c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff7affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7c0d1090 0x7eff7c0c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40716/0 (socket says 172.21.15.37:40716) 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff7affd640 1 -- 172.21.15.37:0/3818296532 learned_addr learned my addr 172.21.15.37:0/3818296532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff7b7fe640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c0cf990 0x7eff7c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff7bfff640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff7c15d320 0x7eff7c153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff7affd640 1 -- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff7c15d320 msgr2=0x7eff7c153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff7affd640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff7c15d320 0x7eff7c153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff7affd640 1 -- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c0cf990 msgr2=0x7eff7c0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff7affd640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c0cf990 0x7eff7c0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff7affd640 1 -- 172.21.15.37:0/3818296532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7c0c8740 con 0x7eff7c0d1090 2024-04-26T20:26:23.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff7affd640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7c0d1090 0x7eff7c0c7fd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7eff7000d800 tx=0x7eff7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff63fff640 1 -- 172.21.15.37:0/3818296532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7000b6c0 con 0x7eff7c0d1090 2024-04-26T20:26:23.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff63fff640 1 -- 172.21.15.37:0/3818296532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff70019070 con 0x7eff7c0d1090 2024-04-26T20:26:23.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff81d11640 1 -- 172.21.15.37:0/3818296532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7c0c8a20 con 0x7eff7c0d1090 2024-04-26T20:26:23.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff63fff640 1 -- 172.21.15.37:0/3818296532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7000ba80 con 0x7eff7c0d1090 2024-04-26T20:26:23.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.682+0000 7eff81d11640 1 -- 172.21.15.37:0/3818296532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff7c0c8fe0 con 0x7eff7c0d1090 2024-04-26T20:26:23.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.683+0000 7eff81d11640 1 -- 172.21.15.37:0/3818296532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff7c0693e0 con 0x7eff7c0d1090 2024-04-26T20:26:23.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.684+0000 7eff63fff640 1 -- 172.21.15.37:0/3818296532 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7eff7000bc20 con 0x7eff7c0d1090 2024-04-26T20:26:23.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.684+0000 7eff63fff640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7eff6806cbc0 0x7eff6806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.684+0000 7eff63fff640 1 -- 172.21.15.37:0/3818296532 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7eff7008c520 con 0x7eff7c0d1090 2024-04-26T20:26:23.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.684+0000 7eff7b7fe640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7eff6806cbc0 0x7eff6806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.685+0000 7eff7b7fe640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7eff6806cbc0 0x7eff6806f080 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7eff6c034000 tx=0x7eff6c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.688+0000 7eff63fff640 1 -- 172.21.15.37:0/3818296532 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff7005a580 con 0x7eff7c0d1090 2024-04-26T20:26:23.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.871+0000 7eff81d11640 1 -- 172.21.15.37:0/3818296532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff7c0cd670 con 0x7eff7c0d1090 2024-04-26T20:26:23.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.871+0000 7eff63fff640 1 -- 172.21.15.37:0/3818296532 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7eff70059e80 con 0x7eff7c0d1090 2024-04-26T20:26:23.873 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.873+0000 7eff81d11640 1 -- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7eff6806cbc0 msgr2=0x7eff6806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.873+0000 7eff81d11640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7eff6806cbc0 0x7eff6806f080 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7eff6c034000 tx=0x7eff6c02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.873+0000 7eff81d11640 1 -- 172.21.15.37:0/3818296532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7c0d1090 msgr2=0x7eff7c0c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.873+0000 7eff81d11640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7c0d1090 0x7eff7c0c7fd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7eff7000d800 tx=0x7eff7000dcc0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.873+0000 7eff81d11640 1 -- 172.21.15.37:0/3818296532 shutdown_connections 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.873+0000 7eff81d11640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7eff6806cbc0 0x7eff6806f080 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.873+0000 7eff81d11640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff7c15d320 0x7eff7c153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.873+0000 7eff81d11640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7c0d1090 0x7eff7c0c7fd0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.873+0000 7eff81d11640 1 --2- 172.21.15.37:0/3818296532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c0cf990 0x7eff7c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.873+0000 7eff81d11640 1 -- 172.21.15.37:0/3818296532 >> 172.21.15.37:0/3818296532 conn(0x7eff7c0d2f80 msgr2=0x7eff7c075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.874+0000 7eff81d11640 1 -- 172.21.15.37:0/3818296532 shutdown_connections 2024-04-26T20:26:23.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.874+0000 7eff81d11640 1 -- 172.21.15.37:0/3818296532 wait complete. 2024-04-26T20:26:23.885 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:26:23.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.988+0000 7f8ead946640 1 Processor -- start 2024-04-26T20:26:23.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.988+0000 7f8ead946640 1 -- start start 2024-04-26T20:26:23.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.989+0000 7f8ead946640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8ea81610c0 0x7f8ea80d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.989+0000 7f8ead946640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ea8161510 con 0x7f8ea80d2150 2024-04-26T20:26:23.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.989+0000 7f8ead946640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ea80d0930 con 0x7f8ea8160800 2024-04-26T20:26:23.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.989+0000 7f8ead946640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f8ea80d25c0 con 0x7f8ea81610c0 2024-04-26T20:26:23.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.989+0000 7f8ea77fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8ea80d2150 0x7f8ea80d3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34454/0 (socket says 172.21.15.37:34454) 2024-04-26T20:26:23.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.989+0000 7f8ea77fe640 1 -- 172.21.15.37:0/1762332085 learned_addr learned my addr 172.21.15.37:0/1762332085 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:23.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.989+0000 7f8ea67fc640 1 --2- 172.21.15.37:0/1762332085 >> v2:172.21.15.134:3300/0 conn(0x7f8ea81610c0 0x7f8ea80d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.989+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2817247855 0 0) 0x7f8ea8161510 con 0x7f8ea80d2150 2024-04-26T20:26:23.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.989+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e90003520 con 0x7f8ea80d2150 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 129071989 0 0) 0x7f8e90003520 con 0x7f8ea80d2150 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 >> v2:172.21.15.134:3300/0 conn(0x7f8ea81610c0 msgr2=0x7f8ea80d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 --2- 172.21.15.37:0/1762332085 >> v2:172.21.15.134:3300/0 conn(0x7f8ea81610c0 0x7f8ea80d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 >> v1:172.21.15.134:6789/0 conn(0x7f8ea8160800 legacy=0x7f8ea80d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea80d0c00 con 0x7f8ea80d2150 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f8ea00019f0 con 0x7f8ea80d2150 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 >> v1:172.21.15.37:6789/0 conn(0x7f8ea80d2150 legacy=0x7f8ea80d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e900039b0 0x7f8e90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e90004800 0x7f8e90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 0x7f8e9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea80d25c0 con 0x7f8e900051c0 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9000c060 con 0x7f8e900039b0 2024-04-26T20:26:23.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8e9000c260 con 0x7f8e90004800 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea77fe640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 0x7f8e9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea6ffd640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e900039b0 0x7f8e90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea67fc640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e90004800 0x7f8e90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea77fe640 1 -- 172.21.15.37:0/1762332085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e90004800 msgr2=0x7f8e90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea77fe640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e90004800 0x7f8e90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea77fe640 1 -- 172.21.15.37:0/1762332085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e900039b0 msgr2=0x7f8e90003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea77fe640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e900039b0 0x7f8e90003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea77fe640 1 -- 172.21.15.37:0/1762332085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e90003520 con 0x7f8e900051c0 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.990+0000 7f8ea67fc640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e90004800 0x7f8e90004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.991+0000 7f8ea77fe640 1 -- 172.21.15.37:0/1762332085 reap_dead start 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.991+0000 7f8ea77fe640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 0x7f8e9000b9c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000a860 tx=0x7f8e90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d081462c6ffc76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.991+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000a480 con 0x7f8e900051c0 2024-04-26T20:26:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.991+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea002f070 con 0x7f8e900051c0 2024-04-26T20:26:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.991+0000 7f8ea5ffb640 1 -- 172.21.15.37:0/1762332085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea002bca0 con 0x7f8e900051c0 2024-04-26T20:26:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.991+0000 7f8ead946640 1 -- 172.21.15.37:0/1762332085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 msgr2=0x7f8e9000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.991+0000 7f8ead946640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 0x7f8e9000b9c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000a860 tx=0x7f8e90005830 comp rx=0 tx=0).stop 2024-04-26T20:26:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.992+0000 7f8ead946640 1 -- 172.21.15.37:0/1762332085 shutdown_connections 2024-04-26T20:26:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.992+0000 7f8ead946640 1 --2- 172.21.15.37:0/1762332085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 0x7f8e9000b9c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.992+0000 7f8ead946640 1 -- 172.21.15.37:0/1762332085 >> 172.21.15.37:0/1762332085 conn(0x7f8ea809e550 msgr2=0x7f8ea80d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.992+0000 7f8ead946640 1 -- 172.21.15.37:0/1762332085 shutdown_connections 2024-04-26T20:26:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.992+0000 7f8ead946640 1 -- 172.21.15.37:0/1762332085 wait complete. 2024-04-26T20:26:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.993+0000 7f8ead946640 1 Processor -- start 2024-04-26T20:26:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.993+0000 7f8ead946640 1 -- start start 2024-04-26T20:26:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ead946640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 0x7f8ea81f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ead946640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ea81f8860 0x7f8ea80b9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ead946640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea81eb2c0 0x7f8ea81eb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ead946640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea80d2e10 con 0x7f8e900051c0 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ead946640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea80d11c0 con 0x7f8ea81eb2c0 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ead946640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea80d30f0 con 0x7f8ea81f8860 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ea6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 0x7f8ea81f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ea6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 0x7f8ea81f9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40722/0 (socket says 172.21.15.37:40722) 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ea6ffd640 1 -- 172.21.15.37:0/2459865996 learned_addr learned my addr 172.21.15.37:0/2459865996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ea6ffd640 1 -- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ea81f8860 msgr2=0x7f8ea80b9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ea6ffd640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ea81f8860 0x7f8ea80b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ea6ffd640 1 -- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea81eb2c0 msgr2=0x7f8ea81eb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ea6ffd640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea81eb2c0 0x7f8ea81eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ea6ffd640 1 -- 172.21.15.37:0/2459865996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea80cf2b0 con 0x7f8e900051c0 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ea6ffd640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 0x7f8ea81f9c10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800f5f0 tx=0x7f8e9800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8e8ffff640 1 -- 172.21.15.37:0/2459865996 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e980052e0 con 0x7f8e900051c0 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8e8ffff640 1 -- 172.21.15.37:0/2459865996 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e98018070 con 0x7f8e900051c0 2024-04-26T20:26:24.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ead946640 1 -- 172.21.15.37:0/2459865996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea80b9a60 con 0x7f8e900051c0 2024-04-26T20:26:24.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8e8ffff640 1 -- 172.21.15.37:0/2459865996 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98003810 con 0x7f8e900051c0 2024-04-26T20:26:24.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.994+0000 7f8ead946640 1 -- 172.21.15.37:0/2459865996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea80ba020 con 0x7f8e900051c0 2024-04-26T20:26:24.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.995+0000 7f8ead946640 1 -- 172.21.15.37:0/2459865996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea806a360 con 0x7f8e900051c0 2024-04-26T20:26:24.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.996+0000 7f8e8ffff640 1 -- 172.21.15.37:0/2459865996 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f8e980039b0 con 0x7f8e900051c0 2024-04-26T20:26:24.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.996+0000 7f8e8ffff640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8e9406cbc0 0x7f8e9406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.996+0000 7f8e8ffff640 1 -- 172.21.15.37:0/2459865996 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f8e9808be40 con 0x7f8e900051c0 2024-04-26T20:26:24.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.997+0000 7f8ea67fc640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8e9406cbc0 0x7f8e9406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:23.997+0000 7f8ea67fc640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8e9406cbc0 0x7f8e9406f080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ea806c9e0 tx=0x7f8e9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.001+0000 7f8e8ffff640 1 -- 172.21.15.37:0/2459865996 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8e98059ee0 con 0x7f8e900051c0 2024-04-26T20:26:24.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.184+0000 7f8ead946640 1 -- 172.21.15.37:0/2459865996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8ea80d0190 con 0x7f8e900051c0 2024-04-26T20:26:24.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.184+0000 7f8e8ffff640 1 -- 172.21.15.37:0/2459865996 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) ==== 104+0+11 (secure 0 0 0) 0x7f8e980597e0 con 0x7f8e900051c0 2024-04-26T20:26:24.185 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T20:26:24.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 -- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8e9406cbc0 msgr2=0x7f8e9406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8e9406cbc0 0x7f8e9406f080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ea806c9e0 tx=0x7f8e9c008040 comp rx=0 tx=0).stop 2024-04-26T20:26:24.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 -- 172.21.15.37:0/2459865996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 msgr2=0x7f8ea81f9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 0x7f8ea81f9c10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800f5f0 tx=0x7f8e9800fab0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 -- 172.21.15.37:0/2459865996 shutdown_connections 2024-04-26T20:26:24.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea81eb2c0 0x7f8ea81eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f8e9406cbc0 0x7f8e9406f080 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ea81f8860 0x7f8ea80b9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 --2- 172.21.15.37:0/2459865996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e900051c0 0x7f8ea81f9c10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 -- 172.21.15.37:0/2459865996 >> 172.21.15.37:0/2459865996 conn(0x7f8ea809e550 msgr2=0x7f8ea8161970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:24.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 -- 172.21.15.37:0/2459865996 shutdown_connections 2024-04-26T20:26:24.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.186+0000 7f8ead946640 1 -- 172.21.15.37:0/2459865996 wait complete. 2024-04-26T20:26:24.198 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:26:24.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.301+0000 7f7349a62640 1 Processor -- start 2024-04-26T20:26:24.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.301+0000 7f7349a62640 1 -- start start 2024-04-26T20:26:24.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.301+0000 7f7349a62640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f734406f9e0 0x7f73440cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.301+0000 7f7349a62640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f734406e7a0 con 0x7f73440d0800 2024-04-26T20:26:24.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.302+0000 7f7349a62640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f734406ea80 con 0x7f73440cde60 2024-04-26T20:26:24.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.302+0000 7f7349a62640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f734416fa90 con 0x7f734406f9e0 2024-04-26T20:26:24.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.302+0000 7f73437fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f73440d0800 0x7f73440cd750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34460/0 (socket says 172.21.15.37:34460) 2024-04-26T20:26:24.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.302+0000 7f73437fe640 1 -- 172.21.15.37:0/4004843519 learned_addr learned my addr 172.21.15.37:0/4004843519 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:24.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.302+0000 7f7343fff640 1 --2- 172.21.15.37:0/4004843519 >> v2:172.21.15.134:3300/0 conn(0x7f734406f9e0 0x7f73440cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.302+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1495725538 0 0) 0x7f734406e7a0 con 0x7f73440d0800 2024-04-26T20:26:24.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f732c003500 con 0x7f73440d0800 2024-04-26T20:26:24.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4225740148 0 0) 0x7f734406ea80 con 0x7f73440cde60 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f734406e7a0 con 0x7f73440cde60 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2956269640 0 0) 0x7f732c003500 con 0x7f73440d0800 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 >> v2:172.21.15.134:3300/0 conn(0x7f734406f9e0 msgr2=0x7f73440cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 --2- 172.21.15.37:0/4004843519 >> v2:172.21.15.134:3300/0 conn(0x7f734406f9e0 0x7f73440cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 >> v1:172.21.15.134:6789/0 conn(0x7f73440cde60 legacy=0x7f73440cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73440cb610 con 0x7f73440d0800 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f7334001670 con 0x7f73440d0800 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 >> v1:172.21.15.37:6789/0 conn(0x7f73440d0800 legacy=0x7f73440cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 0x7f732c0040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f732c004610 0x7f732c00d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f732c00d950 0x7f732c00fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f734416fa90 con 0x7f732c003cd0 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f732c010580 con 0x7f732c00d950 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.339+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f732c010780 con 0x7f732c004610 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f73437fe640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 0x7f732c0040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f7342ffd640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f732c004610 0x7f732c00d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f7343fff640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f732c00d950 0x7f732c00fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f73437fe640 1 -- 172.21.15.37:0/4004843519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f732c004610 msgr2=0x7f732c00d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f73437fe640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f732c004610 0x7f732c00d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f73437fe640 1 -- 172.21.15.37:0/4004843519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f732c00d950 msgr2=0x7f732c00fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f73437fe640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f732c00d950 0x7f732c00fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f73437fe640 1 -- 172.21.15.37:0/4004843519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c003500 con 0x7f732c003cd0 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f7343fff640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f732c00d950 0x7f732c00fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f7342ffd640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f732c004610 0x7f732c00d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f7343fff640 1 -- 172.21.15.37:0/4004843519 reap_dead start 2024-04-26T20:26:24.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.340+0000 7f73437fe640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 0x7f732c0040d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f733400c210 tx=0x7f732c004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559655d6ab63338a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.341+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334007ce0 con 0x7f732c003cd0 2024-04-26T20:26:24.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.341+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7334005ad0 con 0x7f732c003cd0 2024-04-26T20:26:24.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.341+0000 7f73427fc640 1 -- 172.21.15.37:0/4004843519 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334007070 con 0x7f732c003cd0 2024-04-26T20:26:24.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.341+0000 7f7349a62640 1 -- 172.21.15.37:0/4004843519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 msgr2=0x7f732c0040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.341+0000 7f7349a62640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 0x7f732c0040d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f733400c210 tx=0x7f732c004d50 comp rx=0 tx=0).stop 2024-04-26T20:26:24.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.341+0000 7f7349a62640 1 -- 172.21.15.37:0/4004843519 shutdown_connections 2024-04-26T20:26:24.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.341+0000 7f7349a62640 1 --2- 172.21.15.37:0/4004843519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 0x7f732c0040d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.341+0000 7f7349a62640 1 -- 172.21.15.37:0/4004843519 >> 172.21.15.37:0/4004843519 conn(0x7f73440b28c0 msgr2=0x7f73440b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:24.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.341+0000 7f7349a62640 1 -- 172.21.15.37:0/4004843519 shutdown_connections 2024-04-26T20:26:24.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.341+0000 7f7349a62640 1 -- 172.21.15.37:0/4004843519 wait complete. 2024-04-26T20:26:24.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.342+0000 7f7349a62640 1 Processor -- start 2024-04-26T20:26:24.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.343+0000 7f7349a62640 1 -- start start 2024-04-26T20:26:24.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.343+0000 7f7349a62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 0x7f734408c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.343+0000 7f7349a62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73440ca3e0 0x7f73440c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.343+0000 7f7349a62640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73440b54e0 0x7f73440b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.343+0000 7f7349a62640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f734406fe10 con 0x7f732c003cd0 2024-04-26T20:26:24.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.343+0000 7f7349a62640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73440cb3c0 con 0x7f73440ca3e0 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.343+0000 7f7349a62640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f73440cd030 con 0x7f73440b54e0 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f73437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 0x7f734408c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f73437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 0x7f734408c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40728/0 (socket says 172.21.15.37:40728) 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f73437fe640 1 -- 172.21.15.37:0/484146802 learned_addr learned my addr 172.21.15.37:0/484146802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f7342ffd640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73440ca3e0 0x7f73440c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f7343fff640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73440b54e0 0x7f73440b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f73437fe640 1 -- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73440b54e0 msgr2=0x7f73440b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f73437fe640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73440b54e0 0x7f73440b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f73437fe640 1 -- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73440ca3e0 msgr2=0x7f73440c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f73437fe640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73440ca3e0 0x7f73440c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f73437fe640 1 -- 172.21.15.37:0/484146802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7344202470 con 0x7f732c003cd0 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f73437fe640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 0x7f734408c340 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f733400c210 tx=0x7f73440cb7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f732bfff640 1 -- 172.21.15.37:0/484146802 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334037040 con 0x7f732c003cd0 2024-04-26T20:26:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f732bfff640 1 -- 172.21.15.37:0/484146802 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7334041070 con 0x7f732c003cd0 2024-04-26T20:26:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f7349a62640 1 -- 172.21.15.37:0/484146802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73440c8f50 con 0x7f732c003cd0 2024-04-26T20:26:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f732bfff640 1 -- 172.21.15.37:0/484146802 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334001670 con 0x7f732c003cd0 2024-04-26T20:26:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.344+0000 7f7349a62640 1 -- 172.21.15.37:0/484146802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73440c9510 con 0x7f732c003cd0 2024-04-26T20:26:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.345+0000 7f7349a62640 1 -- 172.21.15.37:0/484146802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f734406a360 con 0x7f732c003cd0 2024-04-26T20:26:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.346+0000 7f732bfff640 1 -- 172.21.15.37:0/484146802 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f73340047e0 con 0x7f732c003cd0 2024-04-26T20:26:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.346+0000 7f732bfff640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f733006cbc0 0x7f733006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.346+0000 7f732bfff640 1 -- 172.21.15.37:0/484146802 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f73340ab920 con 0x7f732c003cd0 2024-04-26T20:26:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.346+0000 7f7342ffd640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f733006cbc0 0x7f733006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.347+0000 7f7342ffd640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f733006cbc0 0x7f733006f080 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f73380010e0 tx=0x7f7338009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.351+0000 7f732bfff640 1 -- 172.21.15.37:0/484146802 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7334044370 con 0x7f732c003cd0 2024-04-26T20:26:24.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.534+0000 7f7349a62640 1 -- 172.21.15.37:0/484146802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f73440d1bc0 con 0x7f732c003cd0 2024-04-26T20:26:24.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.534+0000 7f732bfff640 1 -- 172.21.15.37:0/484146802 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) ==== 100+0+11 (secure 0 0 0) 0x7f7334079980 con 0x7f732c003cd0 2024-04-26T20:26:24.535 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T20:26:24.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 -- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f733006cbc0 msgr2=0x7f733006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f733006cbc0 0x7f733006f080 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f73380010e0 tx=0x7f7338009040 comp rx=0 tx=0).stop 2024-04-26T20:26:24.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 -- 172.21.15.37:0/484146802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 msgr2=0x7f734408c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 0x7f734408c340 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f733400c210 tx=0x7f73440cb7e0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 -- 172.21.15.37:0/484146802 shutdown_connections 2024-04-26T20:26:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f733006cbc0 0x7f733006f080 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73440b54e0 0x7f73440b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73440ca3e0 0x7f73440c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 --2- 172.21.15.37:0/484146802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732c003cd0 0x7f734408c340 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 -- 172.21.15.37:0/484146802 >> 172.21.15.37:0/484146802 conn(0x7f73440b28c0 msgr2=0x7f73440ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 -- 172.21.15.37:0/484146802 shutdown_connections 2024-04-26T20:26:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.536+0000 7f7349a62640 1 -- 172.21.15.37:0/484146802 wait complete. 2024-04-26T20:26:24.548 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:24.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.647+0000 7fcfc839d640 1 Processor -- start 2024-04-26T20:26:24.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.647+0000 7fcfc839d640 1 -- start start 2024-04-26T20:26:24.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.648+0000 7fcfc839d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcfc00d3820 0x7fcfc00d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.648+0000 7fcfc839d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc00d0210 con 0x7fcfc0162b40 2024-04-26T20:26:24.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.648+0000 7fcfc839d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc00d04f0 con 0x7fcfc00b4ba0 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.648+0000 7fcfc839d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc00d41c0 con 0x7fcfc00d3820 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.648+0000 7fcfc6139640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fcfc0162b40 0x7fcfc00d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34476/0 (socket says 172.21.15.37:34476) 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.648+0000 7fcfc6139640 1 -- 172.21.15.37:0/766439965 learned_addr learned my addr 172.21.15.37:0/766439965 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3909128904 0 0) 0x7fcfc00d0210 con 0x7fcfc0162b40 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfac003520 con 0x7fcfc0162b40 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5938640 1 --2- 172.21.15.37:0/766439965 >> v2:172.21.15.134:3300/0 conn(0x7fcfc00d3820 0x7fcfc00d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1147064602 0 0) 0x7fcfac003520 con 0x7fcfc0162b40 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 >> v2:172.21.15.134:3300/0 conn(0x7fcfc00d3820 msgr2=0x7fcfc00d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 --2- 172.21.15.37:0/766439965 >> v2:172.21.15.134:3300/0 conn(0x7fcfc00d3820 0x7fcfc00d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 >> v1:172.21.15.134:6789/0 conn(0x7fcfc00b4ba0 legacy=0x7fcfc00b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc00d07d0 con 0x7fcfc0162b40 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fcfbc001670 con 0x7fcfc0162b40 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 >> v1:172.21.15.37:6789/0 conn(0x7fcfc0162b40 legacy=0x7fcfc00d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfac0039b0 0x7fcfac003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcfac004800 0x7fcfac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 0x7fcfac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac00c0c0 con 0x7fcfac0051c0 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac00c270 con 0x7fcfac0039b0 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcfac00c470 con 0x7fcfac004800 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc693a640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 0x7fcfac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc6139640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfac0039b0 0x7fcfac003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc5938640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcfac004800 0x7fcfac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc693a640 1 -- 172.21.15.37:0/766439965 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcfac004800 msgr2=0x7fcfac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc693a640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcfac004800 0x7fcfac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc693a640 1 -- 172.21.15.37:0/766439965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfac0039b0 msgr2=0x7fcfac003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc693a640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfac0039b0 0x7fcfac003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc693a640 1 -- 172.21.15.37:0/766439965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac003520 con 0x7fcfac0051c0 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.649+0000 7fcfc6139640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfac0039b0 0x7fcfac003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.650+0000 7fcfc693a640 1 -- 172.21.15.37:0/766439965 reap_dead start 2024-04-26T20:26:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.650+0000 7fcfc693a640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 0x7fcfac00b9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8004480 tx=0x7fcfac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d4d53bf6a5af3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.650+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8004080 con 0x7fcfac0051c0 2024-04-26T20:26:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.650+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb802ad80 con 0x7fcfac0051c0 2024-04-26T20:26:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.650+0000 7fcfc5137640 1 -- 172.21.15.37:0/766439965 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8006d90 con 0x7fcfac0051c0 2024-04-26T20:26:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.651+0000 7fcfc839d640 1 -- 172.21.15.37:0/766439965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 msgr2=0x7fcfac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.651+0000 7fcfc839d640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 0x7fcfac00b9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8004480 tx=0x7fcfac005830 comp rx=0 tx=0).stop 2024-04-26T20:26:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.651+0000 7fcfc839d640 1 -- 172.21.15.37:0/766439965 shutdown_connections 2024-04-26T20:26:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.651+0000 7fcfc839d640 1 --2- 172.21.15.37:0/766439965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 0x7fcfac00b9c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.651+0000 7fcfc839d640 1 -- 172.21.15.37:0/766439965 >> 172.21.15.37:0/766439965 conn(0x7fcfc009e550 msgr2=0x7fcfc00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.651+0000 7fcfc839d640 1 -- 172.21.15.37:0/766439965 shutdown_connections 2024-04-26T20:26:24.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.651+0000 7fcfc839d640 1 -- 172.21.15.37:0/766439965 wait complete. 2024-04-26T20:26:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.652+0000 7fcfc839d640 1 Processor -- start 2024-04-26T20:26:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.652+0000 7fcfc839d640 1 -- start start 2024-04-26T20:26:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc839d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 0x7fcfc00cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc839d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfc01f4320 0x7fcfc00cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc839d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcfc00cdd10 0x7fcfc006e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc839d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc00d12b0 con 0x7fcfac0051c0 2024-04-26T20:26:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc839d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc00d0d90 con 0x7fcfc01f4320 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc839d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc00cfe30 con 0x7fcfc00cdd10 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc6139640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 0x7fcfc00cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc6139640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 0x7fcfc00cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40748/0 (socket says 172.21.15.37:40748) 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc6139640 1 -- 172.21.15.37:0/2021575973 learned_addr learned my addr 172.21.15.37:0/2021575973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc693a640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcfc00cdd10 0x7fcfc006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc5938640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfc01f4320 0x7fcfc00cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc6139640 1 -- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcfc00cdd10 msgr2=0x7fcfc006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc6139640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcfc00cdd10 0x7fcfc006e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc6139640 1 -- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfc01f4320 msgr2=0x7fcfc00cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc6139640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfc01f4320 0x7fcfc00cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc6139640 1 -- 172.21.15.37:0/2021575973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc00ce190 con 0x7fcfac0051c0 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc6139640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 0x7fcfc00cd0d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00e780 tx=0x7fcfbc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfb6ffd640 1 -- 172.21.15.37:0/2021575973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc00ee90 con 0x7fcfac0051c0 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfb6ffd640 1 -- 172.21.15.37:0/2021575973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfbc018070 con 0x7fcfac0051c0 2024-04-26T20:26:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.653+0000 7fcfc839d640 1 -- 172.21.15.37:0/2021575973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc00cc310 con 0x7fcfac0051c0 2024-04-26T20:26:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.654+0000 7fcfb6ffd640 1 -- 172.21.15.37:0/2021575973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc00caf0 con 0x7fcfac0051c0 2024-04-26T20:26:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.654+0000 7fcfc839d640 1 -- 172.21.15.37:0/2021575973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc00cc8d0 con 0x7fcfac0051c0 2024-04-26T20:26:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.655+0000 7fcfc839d640 1 -- 172.21.15.37:0/2021575973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc006a360 con 0x7fcfac0051c0 2024-04-26T20:26:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.655+0000 7fcfb6ffd640 1 -- 172.21.15.37:0/2021575973 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fcfbc00cc90 con 0x7fcfac0051c0 2024-04-26T20:26:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.656+0000 7fcfb6ffd640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcfa406cd60 0x7fcfa406f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.656+0000 7fcfb6ffd640 1 -- 172.21.15.37:0/2021575973 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fcfbc08bf90 con 0x7fcfac0051c0 2024-04-26T20:26:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.656+0000 7fcfc5938640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcfa406cd60 0x7fcfa406f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.656+0000 7fcfc5938640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcfa406cd60 0x7fcfa406f220 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00d3120 tx=0x7fcfb0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.660+0000 7fcfb6ffd640 1 -- 172.21.15.37:0/2021575973 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcfbc059ff0 con 0x7fcfac0051c0 2024-04-26T20:26:24.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.886+0000 7fcfc839d640 1 -- 172.21.15.37:0/2021575973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfc00d4550 con 0x7fcfac0051c0 2024-04-26T20:26:24.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.887+0000 7fcfb6ffd640 1 -- 172.21.15.37:0/2021575973 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5178 (secure 0 0 0) 0x7fcfbc0598f0 con 0x7fcfac0051c0 2024-04-26T20:26:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/689917778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":689917778},{"type":"v1","addr":"172.21.15.134:6836","nonce":689917778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5362,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/497972484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":497972484},{"type":"v1","addr":"172.21.15.134:6837","nonce":497972484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/221502372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":221502372},{"type":"v1","addr":"172.21.15.37:6833","nonce":221502372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:24:09.772695+0000","modified":"2024-04-26T20:24:11.765632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1915220099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1915220099},{"type":"v1","addr":"172.21.15.37:6835","nonce":1915220099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5688,"qdb_cluster":[5688]},"id":3}]} 2024-04-26T20:26:24.889 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-04-26T20:26:24.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 -- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcfa406cd60 msgr2=0x7fcfa406f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcfa406cd60 0x7fcfa406f220 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00d3120 tx=0x7fcfb0008040 comp rx=0 tx=0).stop 2024-04-26T20:26:24.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 -- 172.21.15.37:0/2021575973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 msgr2=0x7fcfc00cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 0x7fcfc00cd0d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00e780 tx=0x7fcfbc00ec40 comp rx=0 tx=0).stop 2024-04-26T20:26:24.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 -- 172.21.15.37:0/2021575973 shutdown_connections 2024-04-26T20:26:24.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fcfa406cd60 0x7fcfa406f220 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcfc00cdd10 0x7fcfc006e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfc01f4320 0x7fcfc00cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 --2- 172.21.15.37:0/2021575973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfac0051c0 0x7fcfc00cd0d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 -- 172.21.15.37:0/2021575973 >> 172.21.15.37:0/2021575973 conn(0x7fcfc009e550 msgr2=0x7fcfc0075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:24.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 -- 172.21.15.37:0/2021575973 shutdown_connections 2024-04-26T20:26:24.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:24.889+0000 7fcfc839d640 1 -- 172.21.15.37:0/2021575973 wait complete. 2024-04-26T20:26:24.902 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:25.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.001+0000 7fbf7d151640 1 Processor -- start 2024-04-26T20:26:25.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.001+0000 7fbf7d151640 1 -- start start 2024-04-26T20:26:25.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.002+0000 7fbf7d151640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbf78161200 0x7fbf780b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.002+0000 7fbf7d151640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf78161b80 con 0x7fbf78160d70 2024-04-26T20:26:25.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.002+0000 7fbf7d151640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf780b4400 con 0x7fbf7806f790 2024-04-26T20:26:25.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.002+0000 7fbf7d151640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fbf780b4120 con 0x7fbf78161200 2024-04-26T20:26:25.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.002+0000 7fbf7759e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbf78161200 0x7fbf780b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.002+0000 7fbf6e59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbf78160d70 0x7fbf780b5b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34482/0 (socket says 172.21.15.37:34482) 2024-04-26T20:26:25.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.002+0000 7fbf6e59c640 1 -- 172.21.15.37:0/898720908 learned_addr learned my addr 172.21.15.37:0/898720908 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:25.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.002+0000 7fbf7759e640 1 -- 172.21.15.37:0/898720908 >> v1:172.21.15.134:6789/0 conn(0x7fbf7806f790 legacy=0x7fbf78074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:25.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.002+0000 7fbf7759e640 1 -- 172.21.15.37:0/898720908 >> v1:172.21.15.37:6789/0 conn(0x7fbf78160d70 legacy=0x7fbf780b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.002+0000 7fbf7659c640 1 -- 172.21.15.37:0/898720908 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297422686 0 0) 0x7fbf78161b80 con 0x7fbf78160d70 2024-04-26T20:26:25.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.002+0000 7fbf7759e640 1 -- 172.21.15.37:0/898720908 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf78072ff0 con 0x7fbf78161200 2024-04-26T20:26:25.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.003+0000 7fbf7759e640 1 --2- 172.21.15.37:0/898720908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf78161200 0x7fbf780b6210 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009d90 tx=0x7fbf7002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66e5574f1a2ff43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.003+0000 7fbf7659c640 1 -- 172.21.15.37:0/898720908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70007740 con 0x7fbf78161200 2024-04-26T20:26:25.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.003+0000 7fbf7659c640 1 -- 172.21.15.37:0/898720908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf700078e0 con 0x7fbf78161200 2024-04-26T20:26:25.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.003+0000 7fbf7659c640 1 -- 172.21.15.37:0/898720908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70007cb0 con 0x7fbf78161200 2024-04-26T20:26:25.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.003+0000 7fbf7d151640 1 -- 172.21.15.37:0/898720908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf78161200 msgr2=0x7fbf780b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.003+0000 7fbf7d151640 1 --2- 172.21.15.37:0/898720908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf78161200 0x7fbf780b6210 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009d90 tx=0x7fbf7002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.003+0000 7fbf7d151640 1 -- 172.21.15.37:0/898720908 shutdown_connections 2024-04-26T20:26:25.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.003+0000 7fbf7d151640 1 --2- 172.21.15.37:0/898720908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf78161200 0x7fbf780b6210 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.003+0000 7fbf7d151640 1 -- 172.21.15.37:0/898720908 >> 172.21.15.37:0/898720908 conn(0x7fbf780d0d50 msgr2=0x7fbf780d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:25.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.004+0000 7fbf7d151640 1 -- 172.21.15.37:0/898720908 shutdown_connections 2024-04-26T20:26:25.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.004+0000 7fbf7d151640 1 -- 172.21.15.37:0/898720908 wait complete. 2024-04-26T20:26:25.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.005+0000 7fbf7d151640 1 Processor -- start 2024-04-26T20:26:25.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.005+0000 7fbf7d151640 1 -- start start 2024-04-26T20:26:25.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf7d151640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7806f790 0x7fbf78140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf7d151640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf78160d70 0x7fbf781565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf7d151640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf78161200 0x7fbf78158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf7d151640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbf78161650 con 0x7fbf7806f790 2024-04-26T20:26:25.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf7d151640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbf780735b0 con 0x7fbf78161200 2024-04-26T20:26:25.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf7d151640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbf780b3d80 con 0x7fbf78160d70 2024-04-26T20:26:25.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf76d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7806f790 0x7fbf78140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf76d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7806f790 0x7fbf78140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40766/0 (socket says 172.21.15.37:40766) 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf76d9d640 1 -- 172.21.15.37:0/1448001647 learned_addr learned my addr 172.21.15.37:0/1448001647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf7759e640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf78161200 0x7fbf78158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf76d9d640 1 -- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf78160d70 msgr2=0x7fbf781565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf6e59c640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf78160d70 0x7fbf781565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf76d9d640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf78160d70 0x7fbf781565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf76d9d640 1 -- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf78161200 msgr2=0x7fbf78158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf76d9d640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf78161200 0x7fbf78158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf76d9d640 1 -- 172.21.15.37:0/1448001647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf78156b00 con 0x7fbf7806f790 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf7759e640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf78161200 0x7fbf78158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf6e59c640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf78160d70 0x7fbf781565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf76d9d640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7806f790 0x7fbf78140530 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800b420 tx=0x7fbf6800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf6ffff640 1 -- 172.21.15.37:0/1448001647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68011830 con 0x7fbf7806f790 2024-04-26T20:26:25.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.006+0000 7fbf6ffff640 1 -- 172.21.15.37:0/1448001647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf68016070 con 0x7fbf7806f790 2024-04-26T20:26:25.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.007+0000 7fbf7d151640 1 -- 172.21.15.37:0/1448001647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf78140bb0 con 0x7fbf7806f790 2024-04-26T20:26:25.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.007+0000 7fbf6ffff640 1 -- 172.21.15.37:0/1448001647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68011c10 con 0x7fbf7806f790 2024-04-26T20:26:25.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.007+0000 7fbf7d151640 1 -- 172.21.15.37:0/1448001647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf78141170 con 0x7fbf7806f790 2024-04-26T20:26:25.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.007+0000 7fbf7d151640 1 -- 172.21.15.37:0/1448001647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf780b46f0 con 0x7fbf7806f790 2024-04-26T20:26:25.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.008+0000 7fbf6ffff640 1 -- 172.21.15.37:0/1448001647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fbf6801b460 con 0x7fbf7806f790 2024-04-26T20:26:25.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.008+0000 7fbf6ffff640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbf5806cbf0 0x7fbf5806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.009+0000 7fbf6ffff640 1 -- 172.21.15.37:0/1448001647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fbf6808bac0 con 0x7fbf7806f790 2024-04-26T20:26:25.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.009+0000 7fbf6e59c640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbf5806cbf0 0x7fbf5806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.009+0000 7fbf6e59c640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbf5806cbf0 0x7fbf5806f0b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbf7813fa50 tx=0x7fbf60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.013+0000 7fbf6ffff640 1 -- 172.21.15.37:0/1448001647 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf780b46f0 con 0x7fbf7806f790 2024-04-26T20:26:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.202+0000 7fbf7d151640 1 -- 172.21.15.37:0/1448001647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf780b5230 con 0x7fbf7806f790 2024-04-26T20:26:25.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.204+0000 7fbf6ffff640 1 -- 172.21.15.37:0/1448001647 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16715 (secure 0 0 0) 0x7fbf68020020 con 0x7fbf7806f790 2024-04-26T20:26:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:25.256 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":48,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:24:31.476288+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:24:08.047629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:24:08.708738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.f","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:25.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 -- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbf5806cbf0 msgr2=0x7fbf5806f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbf5806cbf0 0x7fbf5806f0b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbf7813fa50 tx=0x7fbf60009040 comp rx=0 tx=0).stop 2024-04-26T20:26:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 -- 172.21.15.37:0/1448001647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7806f790 msgr2=0x7fbf78140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7806f790 0x7fbf78140530 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800b420 tx=0x7fbf6800b8e0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 -- 172.21.15.37:0/1448001647 shutdown_connections 2024-04-26T20:26:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf78161200 0x7fbf78158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fbf5806cbf0 0x7fbf5806f0b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf78160d70 0x7fbf781565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 --2- 172.21.15.37:0/1448001647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7806f790 0x7fbf78140530 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 -- 172.21.15.37:0/1448001647 >> 172.21.15.37:0/1448001647 conn(0x7fbf780d0d50 msgr2=0x7fbf78145390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 -- 172.21.15.37:0/1448001647 shutdown_connections 2024-04-26T20:26:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.206+0000 7fbf7d151640 1 -- 172.21.15.37:0/1448001647 wait complete. 2024-04-26T20:26:25.259 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:26:25.259 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:25.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.360+0000 7fec272ea640 1 Processor -- start 2024-04-26T20:26:25.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.360+0000 7fec272ea640 1 -- start start 2024-04-26T20:26:25.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec272ea640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fec200d22e0 0x7fec200b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec272ea640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fec2015f760 0x7fec200b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec272ea640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec200d3de0 con 0x7fec2015f380 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec272ea640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fec200d43a0 con 0x7fec2015f760 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec272ea640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fec200d40c0 con 0x7fec200d22e0 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec25887640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fec2015f760 0x7fec200b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec25887640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fec2015f760 0x7fec200b8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40770/0 (socket says 172.21.15.37:40770) 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec25887640 1 -- 172.21.15.37:0/2242898629 learned_addr learned my addr 172.21.15.37:0/2242898629 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec24885640 1 --2- 172.21.15.37:0/2242898629 >> v2:172.21.15.134:3300/0 conn(0x7fec200d22e0 0x7fec200b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec25887640 1 -- 172.21.15.37:0/2242898629 >> v2:172.21.15.134:3300/0 conn(0x7fec200d22e0 msgr2=0x7fec200b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec25887640 1 --2- 172.21.15.37:0/2242898629 >> v2:172.21.15.134:3300/0 conn(0x7fec200d22e0 0x7fec200b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec25887640 1 -- 172.21.15.37:0/2242898629 >> v1:172.21.15.134:6789/0 conn(0x7fec2015f380 legacy=0x7fec200d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.361+0000 7fec25887640 1 -- 172.21.15.37:0/2242898629 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec200b8280 con 0x7fec2015f760 2024-04-26T20:26:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.362+0000 7fec25887640 1 --2- 172.21.15.37:0/2242898629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec2015f760 0x7fec200b8fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fec18009e50 tx=0x7fec1802a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ee822e7991c0d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.362+0000 7fec17fff640 1 -- 172.21.15.37:0/2242898629 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18036070 con 0x7fec2015f760 2024-04-26T20:26:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.362+0000 7fec17fff640 1 -- 172.21.15.37:0/2242898629 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec18031600 con 0x7fec2015f760 2024-04-26T20:26:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.362+0000 7fec17fff640 1 -- 172.21.15.37:0/2242898629 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec180319d0 con 0x7fec2015f760 2024-04-26T20:26:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.363+0000 7fec272ea640 1 -- 172.21.15.37:0/2242898629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec2015f760 msgr2=0x7fec200b8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.363+0000 7fec272ea640 1 --2- 172.21.15.37:0/2242898629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec2015f760 0x7fec200b8fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fec18009e50 tx=0x7fec1802a7f0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.363+0000 7fec272ea640 1 -- 172.21.15.37:0/2242898629 shutdown_connections 2024-04-26T20:26:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.363+0000 7fec272ea640 1 --2- 172.21.15.37:0/2242898629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec2015f760 0x7fec200b8fc0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.363+0000 7fec272ea640 1 --2- 172.21.15.37:0/2242898629 >> v2:172.21.15.134:3300/0 conn(0x7fec200d22e0 0x7fec200b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.363+0000 7fec272ea640 1 -- 172.21.15.37:0/2242898629 >> 172.21.15.37:0/2242898629 conn(0x7fec2009e560 msgr2=0x7fec200d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.363+0000 7fec272ea640 1 -- 172.21.15.37:0/2242898629 shutdown_connections 2024-04-26T20:26:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.363+0000 7fec272ea640 1 -- 172.21.15.37:0/2242898629 wait complete. 2024-04-26T20:26:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.364+0000 7fec272ea640 1 Processor -- start 2024-04-26T20:26:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.364+0000 7fec272ea640 1 -- start start 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec272ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec200d22e0 0x7fec201f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec272ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec2015f380 0x7fec2015fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec272ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec2015f760 0x7fec20160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec272ea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec200d1280 con 0x7fec200d22e0 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec272ea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec200d1100 con 0x7fec2015f760 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec272ea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec200d5b50 con 0x7fec2015f380 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec25086640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec200d22e0 0x7fec201f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec25086640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec200d22e0 0x7fec201f4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40780/0 (socket says 172.21.15.37:40780) 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec25086640 1 -- 172.21.15.37:0/4138428720 learned_addr learned my addr 172.21.15.37:0/4138428720 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec25086640 1 -- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec2015f380 msgr2=0x7fec2015fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec25086640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec2015f380 0x7fec2015fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec25086640 1 -- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec2015f760 msgr2=0x7fec20160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec25086640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec2015f760 0x7fec20160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec25086640 1 -- 172.21.15.37:0/4138428720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec200ce200 con 0x7fec200d22e0 2024-04-26T20:26:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.365+0000 7fec25086640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec200d22e0 0x7fec201f4970 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00c400 tx=0x7fec1c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.366+0000 7fec15ffb640 1 -- 172.21.15.37:0/4138428720 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c0052d0 con 0x7fec200d22e0 2024-04-26T20:26:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.366+0000 7fec15ffb640 1 -- 172.21.15.37:0/4138428720 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1c005470 con 0x7fec200d22e0 2024-04-26T20:26:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.366+0000 7fec272ea640 1 -- 172.21.15.37:0/4138428720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec200ce4e0 con 0x7fec200d22e0 2024-04-26T20:26:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.366+0000 7fec15ffb640 1 -- 172.21.15.37:0/4138428720 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c007260 con 0x7fec200d22e0 2024-04-26T20:26:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.366+0000 7fec272ea640 1 -- 172.21.15.37:0/4138428720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec200ceaa0 con 0x7fec200d22e0 2024-04-26T20:26:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.367+0000 7fec272ea640 1 -- 172.21.15.37:0/4138428720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2006a360 con 0x7fec200d22e0 2024-04-26T20:26:25.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.367+0000 7fec15ffb640 1 -- 172.21.15.37:0/4138428720 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fec1c007400 con 0x7fec200d22e0 2024-04-26T20:26:25.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.368+0000 7fec15ffb640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fec0406cbc0 0x7fec0406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.368+0000 7fec15ffb640 1 -- 172.21.15.37:0/4138428720 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fec1c08b660 con 0x7fec200d22e0 2024-04-26T20:26:25.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.368+0000 7fec24885640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fec0406cbc0 0x7fec0406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.369+0000 7fec24885640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fec0406cbc0 0x7fec0406f080 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fec10006a90 tx=0x7fec10006a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.372+0000 7fec15ffb640 1 -- 172.21.15.37:0/4138428720 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec1c059620 con 0x7fec200d22e0 2024-04-26T20:26:25.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.599+0000 7fec272ea640 1 -- 172.21.15.37:0/4138428720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec20145650 con 0x7fec200d22e0 2024-04-26T20:26:25.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.599+0000 7fec15ffb640 1 -- 172.21.15.37:0/4138428720 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5178 (secure 0 0 0) 0x7fec1c058f20 con 0x7fec200d22e0 2024-04-26T20:26:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:25.601 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/689917778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":689917778},{"type":"v1","addr":"172.21.15.134:6836","nonce":689917778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5362,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/497972484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":497972484},{"type":"v1","addr":"172.21.15.134:6837","nonce":497972484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/221502372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":221502372},{"type":"v1","addr":"172.21.15.37:6833","nonce":221502372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:24:09.772695+0000","modified":"2024-04-26T20:24:11.765632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1915220099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1915220099},{"type":"v1","addr":"172.21.15.37:6835","nonce":1915220099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5688,"qdb_cluster":[5688]},"id":3}]} 2024-04-26T20:26:25.601 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-04-26T20:26:25.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 -- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fec0406cbc0 msgr2=0x7fec0406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fec0406cbc0 0x7fec0406f080 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fec10006a90 tx=0x7fec10006a20 comp rx=0 tx=0).stop 2024-04-26T20:26:25.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 -- 172.21.15.37:0/4138428720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec200d22e0 msgr2=0x7fec201f4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec200d22e0 0x7fec201f4970 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00c400 tx=0x7fec1c00c8c0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 -- 172.21.15.37:0/4138428720 shutdown_connections 2024-04-26T20:26:25.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec2015f760 0x7fec20160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fec0406cbc0 0x7fec0406f080 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec2015f380 0x7fec2015fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 --2- 172.21.15.37:0/4138428720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec200d22e0 0x7fec201f4970 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 -- 172.21.15.37:0/4138428720 >> 172.21.15.37:0/4138428720 conn(0x7fec2009e560 msgr2=0x7fec200baad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:25.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 -- 172.21.15.37:0/4138428720 shutdown_connections 2024-04-26T20:26:25.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.602+0000 7fec272ea640 1 -- 172.21.15.37:0/4138428720 wait complete. 2024-04-26T20:26:25.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:25.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.714+0000 7fe7c7a52640 1 Processor -- start 2024-04-26T20:26:25.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.714+0000 7fe7c7a52640 1 -- start start 2024-04-26T20:26:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c7a52640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe7c00d22e0 0x7fe7c00b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c7a52640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7c015f760 0x7fe7c00b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c7a52640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7c00d3de0 con 0x7fe7c015f380 2024-04-26T20:26:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c7a52640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fe7c00d43a0 con 0x7fe7c00d22e0 2024-04-26T20:26:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c7a52640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c00d40c0 con 0x7fe7c015f760 2024-04-26T20:26:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c4fed640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe7c00d22e0 0x7fe7c00b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c57ee640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe7c015f380 0x7fe7c00d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48036/0 (socket says 172.21.15.37:48036) 2024-04-26T20:26:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c57ee640 1 -- 172.21.15.37:0/719004308 learned_addr learned my addr 172.21.15.37:0/719004308 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c5fef640 1 --2- 172.21.15.37:0/719004308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7c015f760 0x7fe7c00b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c4fed640 1 -- 172.21.15.37:0/719004308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7c015f760 msgr2=0x7fe7c00b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c4fed640 1 --2- 172.21.15.37:0/719004308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7c015f760 0x7fe7c00b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c4fed640 1 -- 172.21.15.37:0/719004308 >> v1:172.21.15.37:6789/0 conn(0x7fe7c015f380 legacy=0x7fe7c00d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7c4fed640 1 -- 172.21.15.37:0/719004308 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c00b8280 con 0x7fe7c00d22e0 2024-04-26T20:26:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.715+0000 7fe7b7fff640 1 -- 172.21.15.37:0/719004308 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2770646685 0 0) 0x7fe7c00d3de0 con 0x7fe7c015f380 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.716+0000 7fe7c4fed640 1 --2- 172.21.15.37:0/719004308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c00d22e0 0x7fe7c00b8a80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00097a0 tx=0x7fe7b002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49bf8f705802971 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.716+0000 7fe7b7fff640 1 -- 172.21.15.37:0/719004308 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0037070 con 0x7fe7c00d22e0 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.716+0000 7fe7b7fff640 1 -- 172.21.15.37:0/719004308 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b0032440 con 0x7fe7c00d22e0 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.716+0000 7fe7b7fff640 1 -- 172.21.15.37:0/719004308 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0032810 con 0x7fe7c00d22e0 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.717+0000 7fe7c7a52640 1 -- 172.21.15.37:0/719004308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c00d22e0 msgr2=0x7fe7c00b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.717+0000 7fe7c7a52640 1 --2- 172.21.15.37:0/719004308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c00d22e0 0x7fe7c00b8a80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00097a0 tx=0x7fe7b002a210 comp rx=0 tx=0).stop 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.717+0000 7fe7c7a52640 1 -- 172.21.15.37:0/719004308 shutdown_connections 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.717+0000 7fe7c7a52640 1 --2- 172.21.15.37:0/719004308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7c015f760 0x7fe7c00b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.717+0000 7fe7c7a52640 1 --2- 172.21.15.37:0/719004308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c00d22e0 0x7fe7c00b8a80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.717+0000 7fe7c7a52640 1 -- 172.21.15.37:0/719004308 >> 172.21.15.37:0/719004308 conn(0x7fe7c009e560 msgr2=0x7fe7c00d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.717+0000 7fe7c7a52640 1 -- 172.21.15.37:0/719004308 shutdown_connections 2024-04-26T20:26:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.717+0000 7fe7c7a52640 1 -- 172.21.15.37:0/719004308 wait complete. 2024-04-26T20:26:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.718+0000 7fe7c7a52640 1 Processor -- start 2024-04-26T20:26:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.718+0000 7fe7c7a52640 1 -- start start 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c7a52640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7c00d22e0 0x7fe7c014ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c7a52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c015f380 0x7fe7c00cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c7a52640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7c015f760 0x7fe7c0161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c7a52640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c00d1280 con 0x7fe7c015f380 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c7a52640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c00d1100 con 0x7fe7c015f760 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c7a52640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c00d5b50 con 0x7fe7c00d22e0 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c4fed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c015f380 0x7fe7c00cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c4fed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c015f380 0x7fe7c00cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55744/0 (socket says 172.21.15.37:55744) 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c4fed640 1 -- 172.21.15.37:0/584813437 learned_addr learned my addr 172.21.15.37:0/584813437 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c57ee640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7c00d22e0 0x7fe7c014ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c5fef640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7c015f760 0x7fe7c0161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c4fed640 1 -- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7c00d22e0 msgr2=0x7fe7c014ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c4fed640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7c00d22e0 0x7fe7c014ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c4fed640 1 -- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7c015f760 msgr2=0x7fe7c0161fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c4fed640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7c015f760 0x7fe7c0161fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c4fed640 1 -- 172.21.15.37:0/584813437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c014f2a0 con 0x7fe7c015f380 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c57ee640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7c00d22e0 0x7fe7c014ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c5fef640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7c015f760 0x7fe7c0161fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c4fed640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c015f380 0x7fe7c00cd950 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00098c0 tx=0x7fe7b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7b5ffb640 1 -- 172.21.15.37:0/584813437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0037040 con 0x7fe7c015f380 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7b5ffb640 1 -- 172.21.15.37:0/584813437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b0030770 con 0x7fe7c015f380 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7c7a52640 1 -- 172.21.15.37:0/584813437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c00ce2c0 con 0x7fe7c015f380 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.719+0000 7fe7b5ffb640 1 -- 172.21.15.37:0/584813437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0030b40 con 0x7fe7c015f380 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.720+0000 7fe7c7a52640 1 -- 172.21.15.37:0/584813437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c00ce880 con 0x7fe7c015f380 2024-04-26T20:26:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.721+0000 7fe7c7a52640 1 -- 172.21.15.37:0/584813437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c006a360 con 0x7fe7c015f380 2024-04-26T20:26:25.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.721+0000 7fe7b5ffb640 1 -- 172.21.15.37:0/584813437 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fe7b0044020 con 0x7fe7c015f380 2024-04-26T20:26:25.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.722+0000 7fe7b5ffb640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe7a406cbc0 0x7fe7a406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.722+0000 7fe7b5ffb640 1 -- 172.21.15.37:0/584813437 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe7b00ac490 con 0x7fe7c015f380 2024-04-26T20:26:25.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.722+0000 7fe7c57ee640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe7a406cbc0 0x7fe7a406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.722+0000 7fe7c57ee640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe7a406cbc0 0x7fe7a406f080 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc000f60 tx=0x7fe7bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.726+0000 7fe7b5ffb640 1 -- 172.21.15.37:0/584813437 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe7b0032e20 con 0x7fe7c015f380 2024-04-26T20:26:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.949+0000 7fe7c7a52640 1 -- 172.21.15.37:0/584813437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7c00d0ee0 con 0x7fe7c015f380 2024-04-26T20:26:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.950+0000 7fe7b5ffb640 1 -- 172.21.15.37:0/584813437 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5178 (secure 0 0 0) 0x7fe7b007a530 con 0x7fe7c015f380 2024-04-26T20:26:25.951 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:25.951 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/689917778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":689917778},{"type":"v1","addr":"172.21.15.134:6836","nonce":689917778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5362,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/497972484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":497972484},{"type":"v1","addr":"172.21.15.134:6837","nonce":497972484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/221502372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":221502372},{"type":"v1","addr":"172.21.15.37:6833","nonce":221502372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:24:09.772695+0000","modified":"2024-04-26T20:24:11.765632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1915220099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1915220099},{"type":"v1","addr":"172.21.15.37:6835","nonce":1915220099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5688,"qdb_cluster":[5688]},"id":3}]} 2024-04-26T20:26:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-04-26T20:26:25.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 -- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe7a406cbc0 msgr2=0x7fe7a406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe7a406cbc0 0x7fe7a406f080 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc000f60 tx=0x7fe7bc009040 comp rx=0 tx=0).stop 2024-04-26T20:26:25.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 -- 172.21.15.37:0/584813437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c015f380 msgr2=0x7fe7c00cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c015f380 0x7fe7c00cd950 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00098c0 tx=0x7fe7b00046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 -- 172.21.15.37:0/584813437 shutdown_connections 2024-04-26T20:26:25.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7c015f760 0x7fe7c0161fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c015f380 0x7fe7c00cd950 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fe7a406cbc0 0x7fe7a406f080 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 --2- 172.21.15.37:0/584813437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7c00d22e0 0x7fe7c014ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 -- 172.21.15.37:0/584813437 >> 172.21.15.37:0/584813437 conn(0x7fe7c009e560 msgr2=0x7fe7c00b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:25.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 -- 172.21.15.37:0/584813437 shutdown_connections 2024-04-26T20:26:25.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:25.952+0000 7fe7c7a52640 1 -- 172.21.15.37:0/584813437 wait complete. 2024-04-26T20:26:25.964 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:26.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.063+0000 7fa9af724640 1 Processor -- start 2024-04-26T20:26:26.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.064+0000 7fa9af724640 1 -- start start 2024-04-26T20:26:26.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.064+0000 7fa9af724640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a8162b40 0x7fa9a80d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.064+0000 7fa9af724640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa9a80d3c80 0x7fa9a80d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.064+0000 7fa9af724640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a80d0210 con 0x7fa9a80d3820 2024-04-26T20:26:26.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.064+0000 7fa9af724640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa9a80d04f0 con 0x7fa9a80d3c80 2024-04-26T20:26:26.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.064+0000 7fa9af724640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a80d07d0 con 0x7fa9a8162b40 2024-04-26T20:26:26.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.064+0000 7fa9accbf640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa9a80d3820 0x7fa9a80d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48044/0 (socket says 172.21.15.37:48044) 2024-04-26T20:26:26.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.064+0000 7fa9accbf640 1 -- 172.21.15.37:0/2792172423 learned_addr learned my addr 172.21.15.37:0/2792172423 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:26.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa9adcc1640 1 --2- 172.21.15.37:0/2792172423 >> v2:172.21.15.134:3300/0 conn(0x7fa9a80d3c80 0x7fa9a80d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa9ad4c0640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a8162b40 0x7fa9a80d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2756597148 0 0) 0x7fa9a80d0210 con 0x7fa9a80d3820 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa994003520 con 0x7fa9a80d3820 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3698501590 0 0) 0x7fa994003520 con 0x7fa9a80d3820 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a8162b40 msgr2=0x7fa9a80d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a8162b40 0x7fa9a80d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 >> v2:172.21.15.134:3300/0 conn(0x7fa9a80d3c80 msgr2=0x7fa9a80d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 --2- 172.21.15.37:0/2792172423 >> v2:172.21.15.134:3300/0 conn(0x7fa9a80d3c80 0x7fa9a80d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a80d0ab0 con 0x7fa9a80d3820 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fa998001670 con 0x7fa9a80d3820 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 >> v1:172.21.15.37:6789/0 conn(0x7fa9a80d3820 legacy=0x7fa9a80d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a8162b40 0x7fa9940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 0x7fa994004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa994004d40 0x7fa9940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a80d04f0 con 0x7fa994003f30 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a80d07d0 con 0x7fa994004d40 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa99400b9f0 con 0x7fa9a8162b40 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa9accbf640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 0x7fa994004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa9ad4c0640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a8162b40 0x7fa9940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.065+0000 7fa9adcc1640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa994004d40 0x7fa9940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.066+0000 7fa9accbf640 1 -- 172.21.15.37:0/2792172423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a8162b40 msgr2=0x7fa9940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.066+0000 7fa9accbf640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a8162b40 0x7fa9940039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.066+0000 7fa9accbf640 1 -- 172.21.15.37:0/2792172423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa994004d40 msgr2=0x7fa9940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.066+0000 7fa9accbf640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa994004d40 0x7fa9940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.066+0000 7fa9accbf640 1 -- 172.21.15.37:0/2792172423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa994003520 con 0x7fa994003f30 2024-04-26T20:26:26.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.066+0000 7fa9accbf640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 0x7fa994004800 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa99800c210 tx=0x7fa994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c90acd44a4311e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.066+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998007d50 con 0x7fa994003f30 2024-04-26T20:26:26.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.066+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa998005af0 con 0x7fa994003f30 2024-04-26T20:26:26.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.066+0000 7fa99ffff640 1 -- 172.21.15.37:0/2792172423 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998007070 con 0x7fa994003f30 2024-04-26T20:26:26.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.067+0000 7fa9af724640 1 -- 172.21.15.37:0/2792172423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 msgr2=0x7fa994004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.067+0000 7fa9af724640 1 --2- 172.21.15.37:0/2792172423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 0x7fa994004800 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa99800c210 tx=0x7fa994005830 comp rx=0 tx=0).stop 2024-04-26T20:26:26.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.067+0000 7fa9adcc1640 1 -- 172.21.15.37:0/2792172423 reap_dead start 2024-04-26T20:26:26.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.067+0000 7fa9af724640 1 -- 172.21.15.37:0/2792172423 shutdown_connections 2024-04-26T20:26:26.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.067+0000 7fa9af724640 1 -- 172.21.15.37:0/2792172423 >> 172.21.15.37:0/2792172423 conn(0x7fa9a809e550 msgr2=0x7fa9a80d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:26.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.067+0000 7fa9af724640 1 -- 172.21.15.37:0/2792172423 shutdown_connections 2024-04-26T20:26:26.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.067+0000 7fa9af724640 1 -- 172.21.15.37:0/2792172423 wait complete. 2024-04-26T20:26:26.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.068+0000 7fa9af724640 1 Processor -- start 2024-04-26T20:26:26.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.068+0000 7fa9af724640 1 -- start start 2024-04-26T20:26:26.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9af724640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 0x7fa9a815d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9af724640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a80c9b10 0x7fa9a80ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9af724640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a80c9f90 0x7fa9a80cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9af724640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a80b5bb0 con 0x7fa994003f30 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9af724640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a80d4110 con 0x7fa9a80c9b10 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9af724640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a80d12b0 con 0x7fa9a80c9f90 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9ad4c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 0x7fa9a815d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9ad4c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 0x7fa9a815d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55760/0 (socket says 172.21.15.37:55760) 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9ad4c0640 1 -- 172.21.15.37:0/2133187988 learned_addr learned my addr 172.21.15.37:0/2133187988 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9accbf640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a80c9b10 0x7fa9a80ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9adcc1640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a80c9f90 0x7fa9a80cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9ad4c0640 1 -- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a80c9f90 msgr2=0x7fa9a80cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9ad4c0640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a80c9f90 0x7fa9a80cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9ad4c0640 1 -- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a80c9b10 msgr2=0x7fa9a80ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9ad4c0640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a80c9b10 0x7fa9a80ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.069+0000 7fa9ad4c0640 1 -- 172.21.15.37:0/2133187988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a815dd80 con 0x7fa994003f30 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.070+0000 7fa9ad4c0640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 0x7fa9a815d840 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40119d0 tx=0x7fa9a4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.070+0000 7fa99dffb640 1 -- 172.21.15.37:0/2133187988 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400fb50 con 0x7fa994003f30 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.070+0000 7fa99dffb640 1 -- 172.21.15.37:0/2133187988 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a400fcf0 con 0x7fa994003f30 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.070+0000 7fa9af724640 1 -- 172.21.15.37:0/2133187988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a80c8a70 con 0x7fa994003f30 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.070+0000 7fa99dffb640 1 -- 172.21.15.37:0/2133187988 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400fea0 con 0x7fa994003f30 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.070+0000 7fa9af724640 1 -- 172.21.15.37:0/2133187988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a80c9030 con 0x7fa994003f30 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.071+0000 7fa9af724640 1 -- 172.21.15.37:0/2133187988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a806a360 con 0x7fa994003f30 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.071+0000 7fa99dffb640 1 -- 172.21.15.37:0/2133187988 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fa9a4012040 con 0x7fa994003f30 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.072+0000 7fa99dffb640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa98c06cbf0 0x7fa98c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.072+0000 7fa99dffb640 1 -- 172.21.15.37:0/2133187988 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fa9a408c520 con 0x7fa994003f30 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.072+0000 7fa9accbf640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa98c06cbf0 0x7fa98c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.072+0000 7fa9accbf640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa98c06cbf0 0x7fa98c06f0b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa998033000 tx=0x7fa998009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.076+0000 7fa99dffb640 1 -- 172.21.15.37:0/2133187988 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9a4019030 con 0x7fa994003f30 2024-04-26T20:26:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.261+0000 7fa9af724640 1 -- 172.21.15.37:0/2133187988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9a80d2960 con 0x7fa994003f30 2024-04-26T20:26:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.262+0000 7fa99dffb640 1 -- 172.21.15.37:0/2133187988 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16715 (secure 0 0 0) 0x7fa9a4059fc0 con 0x7fa994003f30 2024-04-26T20:26:26.263 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:26.263 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":48,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:24:31.476288+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:24:08.047629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:24:08.708738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.f","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:26.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 -- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa98c06cbf0 msgr2=0x7fa98c06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa98c06cbf0 0x7fa98c06f0b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa998033000 tx=0x7fa998009030 comp rx=0 tx=0).stop 2024-04-26T20:26:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 -- 172.21.15.37:0/2133187988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 msgr2=0x7fa9a815d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 0x7fa9a815d840 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40119d0 tx=0x7fa9a4011e90 comp rx=0 tx=0).stop 2024-04-26T20:26:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 -- 172.21.15.37:0/2133187988 shutdown_connections 2024-04-26T20:26:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa98c06cbf0 0x7fa98c06f0b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a80c9f90 0x7fa9a80cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a80c9b10 0x7fa9a80ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 --2- 172.21.15.37:0/2133187988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa994003f30 0x7fa9a815d840 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 -- 172.21.15.37:0/2133187988 >> 172.21.15.37:0/2133187988 conn(0x7fa9a809e550 msgr2=0x7fa9a806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 -- 172.21.15.37:0/2133187988 shutdown_connections 2024-04-26T20:26:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.264+0000 7fa9af724640 1 -- 172.21.15.37:0/2133187988 wait complete. 2024-04-26T20:26:26.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:26:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.377+0000 7f3ea8e6b640 1 Processor -- start 2024-04-26T20:26:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.377+0000 7f3ea8e6b640 1 -- start start 2024-04-26T20:26:26.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3ea8e6b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3ea406ea40 0x7f3ea40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3ea8e6b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3ea40cf830 0x7f3ea40cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3ea8e6b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea40d0180 con 0x7f3ea4162e40 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3ea8e6b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea4163220 con 0x7f3ea406ea40 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3ea8e6b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea40d0eb0 con 0x7f3ea40cf830 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3ea259c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3ea4162e40 0x7f3ea40b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48054/0 (socket says 172.21.15.37:48054) 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3ea259c640 1 -- 172.21.15.37:0/4042679499 learned_addr learned my addr 172.21.15.37:0/4042679499 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3e9bfff640 1 --2- 172.21.15.37:0/4042679499 >> v2:172.21.15.37:3300/0 conn(0x7f3ea406ea40 0x7f3ea40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3ea1d9b640 1 -- 172.21.15.37:0/4042679499 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3243599454 0 0) 0x7f3e94001110 con 0x7f3ea4162e40 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3ea1d9b640 1 -- 172.21.15.37:0/4042679499 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e90003520 con 0x7f3ea4162e40 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3ea2d9d640 1 --2- 172.21.15.37:0/4042679499 >> v2:172.21.15.134:3300/0 conn(0x7f3ea40cf830 0x7f3ea40cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3e9bfff640 1 -- 172.21.15.37:0/4042679499 >> v2:172.21.15.134:3300/0 conn(0x7f3ea40cf830 msgr2=0x7f3ea40cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3e9bfff640 1 --2- 172.21.15.37:0/4042679499 >> v2:172.21.15.134:3300/0 conn(0x7f3ea40cf830 0x7f3ea40cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3e9bfff640 1 -- 172.21.15.37:0/4042679499 >> v1:172.21.15.37:6789/0 conn(0x7f3ea4162e40 legacy=0x7f3ea40b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.378+0000 7f3e9bfff640 1 -- 172.21.15.37:0/4042679499 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea40b1d70 con 0x7f3ea406ea40 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.379+0000 7f3e9bfff640 1 --2- 172.21.15.37:0/4042679499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea406ea40 0x7f3ea40d1340 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40737a0 tx=0x7f3e8c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7378149cd3440b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.379+0000 7f3ea1d9b640 1 -- 172.21.15.37:0/4042679499 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c037070 con 0x7f3ea406ea40 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.379+0000 7f3ea1d9b640 1 -- 172.21.15.37:0/4042679499 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8c032440 con 0x7f3ea406ea40 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.379+0000 7f3ea1d9b640 1 -- 172.21.15.37:0/4042679499 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c02ae60 con 0x7f3ea406ea40 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.379+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/4042679499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea406ea40 msgr2=0x7f3ea40d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.379+0000 7f3ea8e6b640 1 --2- 172.21.15.37:0/4042679499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea406ea40 0x7f3ea40d1340 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40737a0 tx=0x7f3e8c02a110 comp rx=0 tx=0).stop 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.380+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/4042679499 shutdown_connections 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.380+0000 7f3ea8e6b640 1 --2- 172.21.15.37:0/4042679499 >> v2:172.21.15.134:3300/0 conn(0x7f3ea40cf830 0x7f3ea40cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.380+0000 7f3ea8e6b640 1 --2- 172.21.15.37:0/4042679499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea406ea40 0x7f3ea40d1340 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.380+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/4042679499 >> 172.21.15.37:0/4042679499 conn(0x7f3ea409ba50 msgr2=0x7f3ea409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.380+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/4042679499 shutdown_connections 2024-04-26T20:26:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.380+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/4042679499 wait complete. 2024-04-26T20:26:26.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.381+0000 7f3ea8e6b640 1 Processor -- start 2024-04-26T20:26:26.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.381+0000 7f3ea8e6b640 1 -- start start 2024-04-26T20:26:26.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea8e6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea406ea40 0x7f3ea40b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea8e6b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ea40cf830 0x7f3ea40b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea8e6b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ea4162e40 0x7f3ea40b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea8e6b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea40d1f20 con 0x7f3ea406ea40 2024-04-26T20:26:26.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea8e6b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea40d1c40 con 0x7f3ea40cf830 2024-04-26T20:26:26.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea8e6b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea40ce010 con 0x7f3ea4162e40 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea406ea40 0x7f3ea40b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea406ea40 0x7f3ea40b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55782/0 (socket says 172.21.15.37:55782) 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea259c640 1 -- 172.21.15.37:0/2804370210 learned_addr learned my addr 172.21.15.37:0/2804370210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3e9bfff640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ea40cf830 0x7f3ea40b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea2d9d640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ea4162e40 0x7f3ea40b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea259c640 1 -- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ea4162e40 msgr2=0x7f3ea40b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea259c640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ea4162e40 0x7f3ea40b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea259c640 1 -- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ea40cf830 msgr2=0x7f3ea40b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea259c640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ea40cf830 0x7f3ea40b4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea259c640 1 -- 172.21.15.37:0/2804370210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea414a1b0 con 0x7f3ea406ea40 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea259c640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea406ea40 0x7f3ea40b4a30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c3a0 tx=0x7f3e9400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3e9affd640 1 -- 172.21.15.37:0/2804370210 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94017070 con 0x7f3ea406ea40 2024-04-26T20:26:26.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3e9affd640 1 -- 172.21.15.37:0/2804370210 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e94005ce0 con 0x7f3ea406ea40 2024-04-26T20:26:26.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/2804370210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea414a490 con 0x7f3ea406ea40 2024-04-26T20:26:26.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3e9affd640 1 -- 172.21.15.37:0/2804370210 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94007310 con 0x7f3ea406ea40 2024-04-26T20:26:26.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.382+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/2804370210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea414aa50 con 0x7f3ea406ea40 2024-04-26T20:26:26.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.383+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/2804370210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea406a360 con 0x7f3ea406ea40 2024-04-26T20:26:26.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.384+0000 7f3e9affd640 1 -- 172.21.15.37:0/2804370210 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f3e9401a070 con 0x7f3ea406ea40 2024-04-26T20:26:26.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.384+0000 7f3e9affd640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e8406cbc0 0x7f3e8406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.384+0000 7f3e9affd640 1 -- 172.21.15.37:0/2804370210 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f3e9408b540 con 0x7f3ea406ea40 2024-04-26T20:26:26.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.384+0000 7f3e9bfff640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e8406cbc0 0x7f3e8406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.385+0000 7f3e9bfff640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e8406cbc0 0x7f3e8406f080 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c034000 tx=0x7f3e8c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.388+0000 7f3e9affd640 1 -- 172.21.15.37:0/2804370210 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ea40d0180 con 0x7f3ea406ea40 2024-04-26T20:26:26.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.616+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/2804370210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3ea40c9e20 con 0x7f3ea406ea40 2024-04-26T20:26:26.618 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-26T20:26:26.617+0000 7f284b532640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:26:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.618+0000 7f3e9affd640 1 -- 172.21.15.37:0/2804370210 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7f3e94059500 con 0x7f3ea406ea40 2024-04-26T20:26:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.622+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e8406cbc0 msgr2=0x7f3e8406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.622+0000 7f3ea8e6b640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e8406cbc0 0x7f3e8406f080 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c034000 tx=0x7f3e8c02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.622+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/2804370210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea406ea40 msgr2=0x7f3ea40b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.622+0000 7f3ea8e6b640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea406ea40 0x7f3ea40b4a30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c3a0 tx=0x7f3e9400c860 comp rx=0 tx=0).stop 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.622+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/2804370210 shutdown_connections 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.622+0000 7f3ea8e6b640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e8406cbc0 0x7f3e8406f080 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.622+0000 7f3ea8e6b640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ea4162e40 0x7f3ea40b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.622+0000 7f3ea8e6b640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ea40cf830 0x7f3ea40b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.622+0000 7f3ea8e6b640 1 --2- 172.21.15.37:0/2804370210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ea406ea40 0x7f3ea40b4a30 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.622+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/2804370210 >> 172.21.15.37:0/2804370210 conn(0x7f3ea409ba50 msgr2=0x7f3ea40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.623+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/2804370210 shutdown_connections 2024-04-26T20:26:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.623+0000 7f3ea8e6b640 1 -- 172.21.15.37:0/2804370210 wait complete. 2024-04-26T20:26:26.626 INFO:tasks.ceph.mds.a.smithi037.stderr: -9241> 2024-04-26T20:24:12.381+0000 7f865e80b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:26:26.635 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:26:26.660 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-26T20:26:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.738+0000 7f11406dc640 1 Processor -- start 2024-04-26T20:26:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.738+0000 7f11406dc640 1 -- start start 2024-04-26T20:26:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f11406dc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f113815f3c0 0x7f113806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f11406dc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f11380cfa50 0x7f113806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f11406dc640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1138074050 con 0x7f11380d1990 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f11406dc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f11380b1060 con 0x7f113815f3c0 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f11406dc640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f11380b1340 con 0x7f11380cfa50 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f113e478640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f113815f3c0 0x7f113806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f113e478640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f113815f3c0 0x7f113806c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55804/0 (socket says 172.21.15.37:55804) 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f113e478640 1 -- 172.21.15.37:0/1968284365 learned_addr learned my addr 172.21.15.37:0/1968284365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f113e478640 1 -- 172.21.15.37:0/1968284365 >> v2:172.21.15.134:3300/0 conn(0x7f11380cfa50 msgr2=0x7f113806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f113ec79640 1 --2- 172.21.15.37:0/1968284365 >> v2:172.21.15.134:3300/0 conn(0x7f11380cfa50 0x7f113806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f113e478640 1 --2- 172.21.15.37:0/1968284365 >> v2:172.21.15.134:3300/0 conn(0x7f11380cfa50 0x7f113806c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f113e478640 1 -- 172.21.15.37:0/1968284365 >> v1:172.21.15.134:6789/0 conn(0x7f11380d1990 legacy=0x7f1138162fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f113e478640 1 -- 172.21.15.37:0/1968284365 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11380b1620 con 0x7f113815f3c0 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.739+0000 7f113ec79640 1 --2- 172.21.15.37:0/1968284365 >> v2:172.21.15.134:3300/0 conn(0x7f11380cfa50 0x7f113806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.740+0000 7f113e478640 1 --2- 172.21.15.37:0/1968284365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113815f3c0 0x7f113806c240 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f11380b7db0 tx=0x7f112802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204a1f12c3541744 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.740+0000 7f113d476640 1 -- 172.21.15.37:0/1968284365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128037070 con 0x7f113815f3c0 2024-04-26T20:26:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.740+0000 7f113d476640 1 -- 172.21.15.37:0/1968284365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1128032440 con 0x7f113815f3c0 2024-04-26T20:26:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.740+0000 7f113d476640 1 -- 172.21.15.37:0/1968284365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128032810 con 0x7f113815f3c0 2024-04-26T20:26:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.741+0000 7f11406dc640 1 -- 172.21.15.37:0/1968284365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113815f3c0 msgr2=0x7f113806c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.741+0000 7f11406dc640 1 --2- 172.21.15.37:0/1968284365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113815f3c0 0x7f113806c240 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f11380b7db0 tx=0x7f112802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.741+0000 7f11406dc640 1 -- 172.21.15.37:0/1968284365 shutdown_connections 2024-04-26T20:26:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.741+0000 7f11406dc640 1 --2- 172.21.15.37:0/1968284365 >> v2:172.21.15.134:3300/0 conn(0x7f11380cfa50 0x7f113806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.741+0000 7f11406dc640 1 --2- 172.21.15.37:0/1968284365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113815f3c0 0x7f113806c240 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.741+0000 7f11406dc640 1 -- 172.21.15.37:0/1968284365 >> 172.21.15.37:0/1968284365 conn(0x7f11380d1e80 msgr2=0x7f11380d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.741+0000 7f11406dc640 1 -- 172.21.15.37:0/1968284365 shutdown_connections 2024-04-26T20:26:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.741+0000 7f11406dc640 1 -- 172.21.15.37:0/1968284365 wait complete. 2024-04-26T20:26:26.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.742+0000 7f11406dc640 1 Processor -- start 2024-04-26T20:26:26.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.742+0000 7f11406dc640 1 -- start start 2024-04-26T20:26:26.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f11406dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11380cfa50 0x7f11381ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f11406dc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11380d1990 0x7f1138147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f11406dc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f113815f3c0 0x7f11381f8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f11406dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11380d1530 con 0x7f11380cfa50 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f11406dc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11380d13b0 con 0x7f11380d1990 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f11406dc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f113815f7a0 con 0x7f113815f3c0 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f113e478640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11380cfa50 0x7f11381ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f113e478640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11380cfa50 0x7f11381ebaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55818/0 (socket says 172.21.15.37:55818) 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f113e478640 1 -- 172.21.15.37:0/2595871527 learned_addr learned my addr 172.21.15.37:0/2595871527 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f113e478640 1 -- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f113815f3c0 msgr2=0x7f11381f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f113e478640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f113815f3c0 0x7f11381f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f113e478640 1 -- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11380d1990 msgr2=0x7f1138147b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f113e478640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11380d1990 0x7f1138147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f113e478640 1 -- 172.21.15.37:0/2595871527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11381f8dc0 con 0x7f11380cfa50 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f113e478640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11380cfa50 0x7f11381ebaa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1128000c00 tx=0x7f11280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f112effd640 1 -- 172.21.15.37:0/2595871527 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128041020 con 0x7f11380cfa50 2024-04-26T20:26:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f112effd640 1 -- 172.21.15.37:0/2595871527 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1128030970 con 0x7f11380cfa50 2024-04-26T20:26:26.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f112effd640 1 -- 172.21.15.37:0/2595871527 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128037040 con 0x7f11380cfa50 2024-04-26T20:26:26.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.743+0000 7f11406dc640 1 -- 172.21.15.37:0/2595871527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1138148190 con 0x7f11380cfa50 2024-04-26T20:26:26.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.744+0000 7f11406dc640 1 -- 172.21.15.37:0/2595871527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1138148750 con 0x7f11380cfa50 2024-04-26T20:26:26.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.744+0000 7f11406dc640 1 -- 172.21.15.37:0/2595871527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113806a360 con 0x7f11380cfa50 2024-04-26T20:26:26.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.745+0000 7f112effd640 1 -- 172.21.15.37:0/2595871527 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f1128044020 con 0x7f11380cfa50 2024-04-26T20:26:26.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.745+0000 7f112effd640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f111c06cbc0 0x7f111c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.745+0000 7f112effd640 1 -- 172.21.15.37:0/2595871527 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5853+0+0 (secure 0 0 0) 0x7f11280ac6b0 con 0x7f11380cfa50 2024-04-26T20:26:26.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.746+0000 7f113dc77640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f111c06cbc0 0x7f111c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.746+0000 7f113dc77640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f111c06cbc0 0x7f111c06f080 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f113814c180 tx=0x7f1134009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.750+0000 7f112effd640 1 -- 172.21.15.37:0/2595871527 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f112804b070 con 0x7f11380cfa50 2024-04-26T20:26:26.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:26.978+0000 7f11406dc640 1 -- 172.21.15.37:0/2595871527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f11380b7e00 con 0x7f11380cfa50 2024-04-26T20:26:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.620+0000 7f112effd640 1 -- 172.21.15.37:0/2595871527 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7f112807a790 con 0x7f11380cfa50 2024-04-26T20:26:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 -- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f111c06cbc0 msgr2=0x7f111c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f111c06cbc0 0x7f111c06f080 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f113814c180 tx=0x7f1134009250 comp rx=0 tx=0).stop 2024-04-26T20:26:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 -- 172.21.15.37:0/2595871527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11380cfa50 msgr2=0x7f11381ebaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11380cfa50 0x7f11381ebaa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1128000c00 tx=0x7f11280046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 -- 172.21.15.37:0/2595871527 shutdown_connections 2024-04-26T20:26:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f111c06cbc0 0x7f111c06f080 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f113815f3c0 0x7f11381f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11380d1990 0x7f1138147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 --2- 172.21.15.37:0/2595871527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11380cfa50 0x7f11381ebaa0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 -- 172.21.15.37:0/2595871527 >> 172.21.15.37:0/2595871527 conn(0x7f11380d1e80 msgr2=0x7f11380b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 -- 172.21.15.37:0/2595871527 shutdown_connections 2024-04-26T20:26:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.623+0000 7f11406dc640 1 -- 172.21.15.37:0/2595871527 wait complete. 2024-04-26T20:26:27.637 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:26:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.789+0000 7f34cf8b0640 1 Processor -- start 2024-04-26T20:26:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.789+0000 7f34cf8b0640 1 -- start start 2024-04-26T20:26:27.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cf8b0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f34c815f410 0x7f34c80b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cf8b0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34c815f7f0 0x7f34c80d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cf8b0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c80b55a0 con 0x7f34c80b50c0 2024-04-26T20:26:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cf8b0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f34c80b5880 con 0x7f34c815f410 2024-04-26T20:26:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cf8b0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f34c80b5b60 con 0x7f34c815f7f0 2024-04-26T20:26:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cd64c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f34c815f410 0x7f34c80b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cde4d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f34c80b50c0 0x7f34c80d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48074/0 (socket says 172.21.15.37:48074) 2024-04-26T20:26:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cde4d640 1 -- 172.21.15.37:0/4074113052 learned_addr learned my addr 172.21.15.37:0/4074113052 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cce4b640 1 --2- 172.21.15.37:0/4074113052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34c815f7f0 0x7f34c80d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cd64c640 1 -- 172.21.15.37:0/4074113052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34c815f7f0 msgr2=0x7f34c80d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cd64c640 1 --2- 172.21.15.37:0/4074113052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34c815f7f0 0x7f34c80d3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cd64c640 1 -- 172.21.15.37:0/4074113052 >> v1:172.21.15.37:6789/0 conn(0x7f34c80b50c0 legacy=0x7f34c80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34cd64c640 1 -- 172.21.15.37:0/4074113052 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c80b5e40 con 0x7f34c815f410 2024-04-26T20:26:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.790+0000 7f34c7fff640 1 -- 172.21.15.37:0/4074113052 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379203047 0 0) 0x7f34c80b55a0 con 0x7f34c80b50c0 2024-04-26T20:26:27.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.791+0000 7f34cd64c640 1 --2- 172.21.15.37:0/4074113052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c815f410 0x7f34c80b45e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009d90 tx=0x7f34b802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1299a3b67f300afc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:27.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.791+0000 7f34c7fff640 1 -- 172.21.15.37:0/4074113052 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8007740 con 0x7f34c815f410 2024-04-26T20:26:27.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.791+0000 7f34c7fff640 1 -- 172.21.15.37:0/4074113052 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b80078e0 con 0x7f34c815f410 2024-04-26T20:26:27.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.791+0000 7f34c7fff640 1 -- 172.21.15.37:0/4074113052 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8007cb0 con 0x7f34c815f410 2024-04-26T20:26:27.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.792+0000 7f34cf8b0640 1 -- 172.21.15.37:0/4074113052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c815f410 msgr2=0x7f34c80b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.792+0000 7f34cf8b0640 1 --2- 172.21.15.37:0/4074113052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c815f410 0x7f34c80b45e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009d90 tx=0x7f34b802a6f0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.792+0000 7f34cf8b0640 1 -- 172.21.15.37:0/4074113052 shutdown_connections 2024-04-26T20:26:27.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.792+0000 7f34cf8b0640 1 --2- 172.21.15.37:0/4074113052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34c815f7f0 0x7f34c80d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.792+0000 7f34cf8b0640 1 --2- 172.21.15.37:0/4074113052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c815f410 0x7f34c80b45e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.792+0000 7f34cf8b0640 1 -- 172.21.15.37:0/4074113052 >> 172.21.15.37:0/4074113052 conn(0x7f34c808ebb0 msgr2=0x7f34c809e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:27.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.792+0000 7f34cf8b0640 1 -- 172.21.15.37:0/4074113052 shutdown_connections 2024-04-26T20:26:27.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.792+0000 7f34cf8b0640 1 -- 172.21.15.37:0/4074113052 wait complete. 2024-04-26T20:26:27.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.793+0000 7f34cf8b0640 1 Processor -- start 2024-04-26T20:26:27.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.793+0000 7f34cf8b0640 1 -- start start 2024-04-26T20:26:27.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cf8b0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34c80b50c0 0x7f34c80c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cf8b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c815f410 0x7f34c8161df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cf8b0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34c8155da0 0x7f34c8151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cf8b0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34c80d12a0 con 0x7f34c815f410 2024-04-26T20:26:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cf8b0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34c80d1120 con 0x7f34c8155da0 2024-04-26T20:26:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cf8b0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f34c80d1420 con 0x7f34c80b50c0 2024-04-26T20:26:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cce4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c815f410 0x7f34c8161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cce4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c815f410 0x7f34c8161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55826/0 (socket says 172.21.15.37:55826) 2024-04-26T20:26:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cce4b640 1 -- 172.21.15.37:0/2155626095 learned_addr learned my addr 172.21.15.37:0/2155626095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cce4b640 1 -- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34c80b50c0 msgr2=0x7f34c80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cce4b640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34c80b50c0 0x7f34c80c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cce4b640 1 -- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34c8155da0 msgr2=0x7f34c8151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cce4b640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34c8155da0 0x7f34c8151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cce4b640 1 -- 172.21.15.37:0/2155626095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c81562f0 con 0x7f34c815f410 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.794+0000 7f34cce4b640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c815f410 0x7f34c8161df0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00d5f0 tx=0x7f34bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.795+0000 7f34c5ffb640 1 -- 172.21.15.37:0/2155626095 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0125f0 con 0x7f34c815f410 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.795+0000 7f34c5ffb640 1 -- 172.21.15.37:0/2155626095 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34bc019070 con 0x7f34c815f410 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.795+0000 7f34cf8b0640 1 -- 172.21.15.37:0/2155626095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c81565d0 con 0x7f34c815f410 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.795+0000 7f34c5ffb640 1 -- 172.21.15.37:0/2155626095 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0129d0 con 0x7f34c815f410 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.795+0000 7f34cf8b0640 1 -- 172.21.15.37:0/2155626095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c8156b90 con 0x7f34c815f410 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.796+0000 7f34cf8b0640 1 -- 172.21.15.37:0/2155626095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c806a360 con 0x7f34c815f410 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.796+0000 7f34c5ffb640 1 -- 172.21.15.37:0/2155626095 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f34bc012b70 con 0x7f34c815f410 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.796+0000 7f34c5ffb640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34b406cbf0 0x7f34b406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.797+0000 7f34c5ffb640 1 -- 172.21.15.37:0/2155626095 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5853+0+0 (secure 0 0 0) 0x7f34bc08c4c0 con 0x7f34c815f410 2024-04-26T20:26:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.797+0000 7f34cd64c640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34b406cbf0 0x7f34b406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.797+0000 7f34cd64c640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34b406cbf0 0x7f34b406f0b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f34b8034000 tx=0x7f34b802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.801+0000 7f34c5ffb640 1 -- 172.21.15.37:0/2155626095 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34bc05a480 con 0x7f34c815f410 2024-04-26T20:26:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:27.986+0000 7f34cf8b0640 1 -- 172.21.15.37:0/2155626095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f34c80d0a60 con 0x7f34c815f410 2024-04-26T20:26:28.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.679+0000 7f34c5ffb640 1 -- 172.21.15.37:0/2155626095 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) ==== 181+0+0 (secure 0 0 0) 0x7f34bc059d80 con 0x7f34c815f410 2024-04-26T20:26:28.680 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:26:28.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.682+0000 7f34cf8b0640 1 -- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34b406cbf0 msgr2=0x7f34b406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.682+0000 7f34cf8b0640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34b406cbf0 0x7f34b406f0b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f34b8034000 tx=0x7f34b802e000 comp rx=0 tx=0).stop 2024-04-26T20:26:28.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.683+0000 7f34cf8b0640 1 -- 172.21.15.37:0/2155626095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c815f410 msgr2=0x7f34c8161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.683+0000 7f34cf8b0640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c815f410 0x7f34c8161df0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00d5f0 tx=0x7f34bc00dab0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.684+0000 7f34cf8b0640 1 -- 172.21.15.37:0/2155626095 shutdown_connections 2024-04-26T20:26:28.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.684+0000 7f34cf8b0640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34c8155da0 0x7f34c8151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.684+0000 7f34cf8b0640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34c815f410 0x7f34c8161df0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.684+0000 7f34cf8b0640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f34b406cbf0 0x7f34b406f0b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.684+0000 7f34cf8b0640 1 --2- 172.21.15.37:0/2155626095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34c80b50c0 0x7f34c80c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.684+0000 7f34cf8b0640 1 -- 172.21.15.37:0/2155626095 >> 172.21.15.37:0/2155626095 conn(0x7f34c808ebb0 msgr2=0x7f34c806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.684+0000 7f34cf8b0640 1 -- 172.21.15.37:0/2155626095 shutdown_connections 2024-04-26T20:26:28.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.684+0000 7f34cf8b0640 1 -- 172.21.15.37:0/2155626095 wait complete. 2024-04-26T20:26:28.699 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:26:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.807+0000 7f9318a46640 1 Processor -- start 2024-04-26T20:26:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.807+0000 7f9318a46640 1 -- start start 2024-04-26T20:26:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.808+0000 7f9318a46640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93140c6ae0 0x7f93140cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.808+0000 7f9318a46640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93140cd5d0 0x7f93140cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.808+0000 7f9318a46640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93140cbc00 con 0x7f9314146800 2024-04-26T20:26:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.808+0000 7f9318a46640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f93140cc1c0 con 0x7f93140cd5d0 2024-04-26T20:26:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.808+0000 7f9318a46640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93140ceef0 con 0x7f93140c6ae0 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.808+0000 7f931259c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9314146800 0x7f9314071f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48082/0 (socket says 172.21.15.37:48082) 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.808+0000 7f931259c640 1 -- 172.21.15.37:0/469263543 learned_addr learned my addr 172.21.15.37:0/469263543 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 850653528 0 0) 0x7f93140cbc00 con 0x7f9314146800 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9300003520 con 0x7f9314146800 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f9312d9d640 1 --2- 172.21.15.37:0/469263543 >> v2:172.21.15.134:3300/0 conn(0x7f93140cd5d0 0x7f93140cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f9311d9b640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93140c6ae0 0x7f93140cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3734072410 0 0) 0x7f9300003520 con 0x7f9314146800 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93140c6ae0 msgr2=0x7f93140cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93140c6ae0 0x7f93140cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 >> v2:172.21.15.134:3300/0 conn(0x7f93140cd5d0 msgr2=0x7f93140cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 --2- 172.21.15.37:0/469263543 >> v2:172.21.15.134:3300/0 conn(0x7f93140cd5d0 0x7f93140cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93140cbee0 con 0x7f9314146800 2024-04-26T20:26:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f930c001670 con 0x7f9314146800 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 >> v1:172.21.15.37:6789/0 conn(0x7f9314146800 legacy=0x7f9314071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93140c6ae0 0x7f93000039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9300004000 0x7f9300004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9300004d40 0x7f93000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f93140ceef0 con 0x7f9300004d40 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f930000ba70 con 0x7f9300004000 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f930000bc70 con 0x7f93140c6ae0 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f9312d9d640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9300004d40 0x7f93000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f9311d9b640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9300004000 0x7f9300004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f931259c640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93140c6ae0 0x7f93000039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f9312d9d640 1 -- 172.21.15.37:0/469263543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93140c6ae0 msgr2=0x7f93000039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f9312d9d640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93140c6ae0 0x7f93000039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f9312d9d640 1 -- 172.21.15.37:0/469263543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9300004000 msgr2=0x7f9300004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f9312d9d640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9300004000 0x7f9300004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.809+0000 7f9312d9d640 1 -- 172.21.15.37:0/469263543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9300003520 con 0x7f9300004d40 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.810+0000 7f9312d9d640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9300004d40 0x7f93000051c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9304007360 tx=0x7f9300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5824fc33e7c0e40b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.810+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304009be0 con 0x7f9300004d40 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.810+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f930402e620 con 0x7f9300004d40 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.810+0000 7f931159a640 1 -- 172.21.15.37:0/469263543 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930402e9f0 con 0x7f9300004d40 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.811+0000 7f9318a46640 1 -- 172.21.15.37:0/469263543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9300004d40 msgr2=0x7f93000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.811+0000 7f9318a46640 1 --2- 172.21.15.37:0/469263543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9300004d40 0x7f93000051c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9304007360 tx=0x7f9300005830 comp rx=0 tx=0).stop 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.811+0000 7f9312d9d640 1 -- 172.21.15.37:0/469263543 reap_dead start 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.811+0000 7f9318a46640 1 -- 172.21.15.37:0/469263543 shutdown_connections 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.811+0000 7f9318a46640 1 -- 172.21.15.37:0/469263543 >> 172.21.15.37:0/469263543 conn(0x7f93140b2c60 msgr2=0x7f93140b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.811+0000 7f9318a46640 1 -- 172.21.15.37:0/469263543 shutdown_connections 2024-04-26T20:26:28.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.811+0000 7f9318a46640 1 -- 172.21.15.37:0/469263543 wait complete. 2024-04-26T20:26:28.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.812+0000 7f9318a46640 1 Processor -- start 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.812+0000 7f9318a46640 1 -- start start 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.813+0000 7f9318a46640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9300004d40 0x7f931415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.813+0000 7f9318a46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f931413b660 0x7f9314072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.813+0000 7f9318a46640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9314073430 0x7f93140ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.813+0000 7f9318a46640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f93140cf420 con 0x7f931413b660 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.813+0000 7f9318a46640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93140c6f40 con 0x7f9300004d40 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.813+0000 7f9318a46640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93140cdf60 con 0x7f9314073430 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.813+0000 7f9311d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f931413b660 0x7f9314072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.813+0000 7f9311d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f931413b660 0x7f9314072ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55838/0 (socket says 172.21.15.37:55838) 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.813+0000 7f9311d9b640 1 -- 172.21.15.37:0/2869213966 learned_addr learned my addr 172.21.15.37:0/2869213966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.813+0000 7f9312d9d640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9314073430 0x7f93140ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.813+0000 7f931259c640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9300004d40 0x7f931415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f9311d9b640 1 -- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9314073430 msgr2=0x7f93140ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f9311d9b640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9314073430 0x7f93140ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f9311d9b640 1 -- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9300004d40 msgr2=0x7f931415ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f9311d9b640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9300004d40 0x7f931415ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f9311d9b640 1 -- 172.21.15.37:0/2869213966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931406e250 con 0x7f931413b660 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f9312d9d640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9314073430 0x7f93140ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f931259c640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9300004d40 0x7f931415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f9311d9b640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f931413b660 0x7f9314072ef0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00b7f0 tx=0x7f92fc00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f930affd640 1 -- 172.21.15.37:0/2869213966 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc018070 con 0x7f931413b660 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f930affd640 1 -- 172.21.15.37:0/2869213966 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92fc0107d0 con 0x7f931413b660 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f9318a46640 1 -- 172.21.15.37:0/2869213966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931406e530 con 0x7f931413b660 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f930affd640 1 -- 172.21.15.37:0/2869213966 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc010ba0 con 0x7f931413b660 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.814+0000 7f9318a46640 1 -- 172.21.15.37:0/2869213966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931406eaf0 con 0x7f931413b660 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.815+0000 7f9318a46640 1 -- 172.21.15.37:0/2869213966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f931406a360 con 0x7f931413b660 2024-04-26T20:26:28.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.815+0000 7f930affd640 1 -- 172.21.15.37:0/2869213966 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f92fc01b070 con 0x7f931413b660 2024-04-26T20:26:28.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.816+0000 7f930affd640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92f406cbc0 0x7f92f406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.816+0000 7f930affd640 1 -- 172.21.15.37:0/2869213966 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5415+0+0 (secure 0 0 0) 0x7f92fc08c7b0 con 0x7f931413b660 2024-04-26T20:26:28.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.816+0000 7f931259c640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92f406cbc0 0x7f92f406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.817+0000 7f931259c640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92f406cbc0 0x7f92f406f080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f930c0017e0 tx=0x7f930c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:28.820+0000 7f930affd640 1 -- 172.21.15.37:0/2869213966 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92fc05a9b0 con 0x7f931413b660 2024-04-26T20:26:29.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.007+0000 7f9318a46640 1 -- 172.21.15.37:0/2869213966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f93140b2660 con 0x7f931413b660 2024-04-26T20:26:29.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.684+0000 7f930affd640 1 -- 172.21.15.37:0/2869213966 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) ==== 169+0+0 (secure 0 0 0) 0x7f92fc05a2b0 con 0x7f931413b660 2024-04-26T20:26:29.685 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-26T20:26:29.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.689+0000 7f9318a46640 1 -- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92f406cbc0 msgr2=0x7f92f406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.689+0000 7f9318a46640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92f406cbc0 0x7f92f406f080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f930c0017e0 tx=0x7f930c009040 comp rx=0 tx=0).stop 2024-04-26T20:26:29.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.689+0000 7f9318a46640 1 -- 172.21.15.37:0/2869213966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f931413b660 msgr2=0x7f9314072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.689+0000 7f9318a46640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f931413b660 0x7f9314072ef0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00b7f0 tx=0x7f92fc00bcb0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.690+0000 7f9318a46640 1 -- 172.21.15.37:0/2869213966 shutdown_connections 2024-04-26T20:26:29.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.690+0000 7f9318a46640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f92f406cbc0 0x7f92f406f080 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.690+0000 7f9318a46640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9314073430 0x7f93140ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.690+0000 7f9318a46640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f931413b660 0x7f9314072ef0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.690+0000 7f9318a46640 1 --2- 172.21.15.37:0/2869213966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9300004d40 0x7f931415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.690+0000 7f9318a46640 1 -- 172.21.15.37:0/2869213966 >> 172.21.15.37:0/2869213966 conn(0x7f93140b2c60 msgr2=0x7f93140c5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.690+0000 7f9318a46640 1 -- 172.21.15.37:0/2869213966 shutdown_connections 2024-04-26T20:26:29.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.691+0000 7f9318a46640 1 -- 172.21.15.37:0/2869213966 wait complete. 2024-04-26T20:26:29.707 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T20:26:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.813+0000 7fa34370a640 1 Processor -- start 2024-04-26T20:26:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.813+0000 7fa34370a640 1 -- start start 2024-04-26T20:26:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.814+0000 7fa34370a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa33c06f3f0 0x7fa33c06e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.814+0000 7fa34370a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c068300 0x7fa33c06ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.814+0000 7fa34370a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa33c0d0fc0 con 0x7fa33c15d140 2024-04-26T20:26:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.814+0000 7fa34370a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa33c0cd750 con 0x7fa33c06f3f0 2024-04-26T20:26:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.814+0000 7fa34370a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c0cdd10 con 0x7fa33c068300 2024-04-26T20:26:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.814+0000 7fa3414a6640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa33c15d140 0x7fa33c06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48086/0 (socket says 172.21.15.37:48086) 2024-04-26T20:26:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.814+0000 7fa3414a6640 1 -- 172.21.15.37:0/2546933509 learned_addr learned my addr 172.21.15.37:0/2546933509 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.814+0000 7fa340ca5640 1 --2- 172.21.15.37:0/2546933509 >> v2:172.21.15.134:3300/0 conn(0x7fa33c06f3f0 0x7fa33c06e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.814+0000 7fa341ca7640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c068300 0x7fa33c06ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.814+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3508939750 0 0) 0x7fa33c0d0fc0 con 0x7fa33c15d140 2024-04-26T20:26:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.814+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa328003520 con 0x7fa33c15d140 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1149424013 0 0) 0x7fa328003520 con 0x7fa33c15d140 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c068300 msgr2=0x7fa33c06ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c068300 0x7fa33c06ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 >> v2:172.21.15.134:3300/0 conn(0x7fa33c06f3f0 msgr2=0x7fa33c06e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 --2- 172.21.15.37:0/2546933509 >> v2:172.21.15.134:3300/0 conn(0x7fa33c06f3f0 0x7fa33c06e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c0d12a0 con 0x7fa33c15d140 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fa334001670 con 0x7fa33c15d140 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 >> v1:172.21.15.37:6789/0 conn(0x7fa33c15d140 legacy=0x7fa33c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c068300 0x7fa3280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 0x7fa328004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa328004d40 0x7fa3280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c0cd750 con 0x7fa328003f30 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c0cdd10 con 0x7fa328004d40 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa32800b9c0 con 0x7fa33c068300 2024-04-26T20:26:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa340ca5640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 0x7fa328004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa3414a6640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c068300 0x7fa3280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa341ca7640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa328004d40 0x7fa3280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa340ca5640 1 -- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c068300 msgr2=0x7fa3280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa340ca5640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c068300 0x7fa3280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa340ca5640 1 -- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa328004d40 msgr2=0x7fa3280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa340ca5640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa328004d40 0x7fa3280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa340ca5640 1 -- 172.21.15.37:0/2546933509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa328003520 con 0x7fa328003f30 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.815+0000 7fa3414a6640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c068300 0x7fa3280039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.816+0000 7fa340ca5640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 0x7fa328004800 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009ac0 tx=0x7fa328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb22ded770389f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.816+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c036070 con 0x7fa328003f30 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.816+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa32c031400 con 0x7fa328003f30 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.816+0000 7fa333fff640 1 -- 172.21.15.37:0/2546933509 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c009690 con 0x7fa328003f30 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.816+0000 7fa34370a640 1 -- 172.21.15.37:0/2546933509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 msgr2=0x7fa328004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.816+0000 7fa34370a640 1 --2- 172.21.15.37:0/2546933509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 0x7fa328004800 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009ac0 tx=0x7fa328005830 comp rx=0 tx=0).stop 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.816+0000 7fa341ca7640 1 -- 172.21.15.37:0/2546933509 reap_dead start 2024-04-26T20:26:29.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.816+0000 7fa34370a640 1 -- 172.21.15.37:0/2546933509 shutdown_connections 2024-04-26T20:26:29.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.816+0000 7fa34370a640 1 -- 172.21.15.37:0/2546933509 >> 172.21.15.37:0/2546933509 conn(0x7fa33c0d16b0 msgr2=0x7fa33c0d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.817+0000 7fa34370a640 1 -- 172.21.15.37:0/2546933509 shutdown_connections 2024-04-26T20:26:29.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.817+0000 7fa34370a640 1 -- 172.21.15.37:0/2546933509 wait complete. 2024-04-26T20:26:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.818+0000 7fa34370a640 1 Processor -- start 2024-04-26T20:26:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.818+0000 7fa34370a640 1 -- start start 2024-04-26T20:26:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.818+0000 7fa34370a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 0x7fa33c14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.818+0000 7fa34370a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa33c15ae40 0x7fa33c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa34370a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c0ae9e0 0x7fa33c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa34370a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c06c950 con 0x7fa328003f30 2024-04-26T20:26:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa34370a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c15d520 con 0x7fa33c15ae40 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa34370a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c0d0b50 con 0x7fa33c0ae9e0 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa3414a6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 0x7fa33c14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa3414a6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 0x7fa33c14c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55858/0 (socket says 172.21.15.37:55858) 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa3414a6640 1 -- 172.21.15.37:0/2436835299 learned_addr learned my addr 172.21.15.37:0/2436835299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa3414a6640 1 -- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c0ae9e0 msgr2=0x7fa33c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa3414a6640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c0ae9e0 0x7fa33c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa340ca5640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa33c15ae40 0x7fa33c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa3414a6640 1 -- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa33c15ae40 msgr2=0x7fa33c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa3414a6640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa33c15ae40 0x7fa33c15b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa3414a6640 1 -- 172.21.15.37:0/2436835299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c0af780 con 0x7fa328003f30 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa3414a6640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 0x7fa33c14c110 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa33400e730 tx=0x7fa33400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa331ffb640 1 -- 172.21.15.37:0/2436835299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33400ee40 con 0x7fa328003f30 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa331ffb640 1 -- 172.21.15.37:0/2436835299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa334018070 con 0x7fa328003f30 2024-04-26T20:26:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa34370a640 1 -- 172.21.15.37:0/2436835299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33c0afa60 con 0x7fa328003f30 2024-04-26T20:26:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.819+0000 7fa331ffb640 1 -- 172.21.15.37:0/2436835299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33400cad0 con 0x7fa328003f30 2024-04-26T20:26:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.820+0000 7fa34370a640 1 -- 172.21.15.37:0/2436835299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c0b0020 con 0x7fa328003f30 2024-04-26T20:26:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.820+0000 7fa34370a640 1 -- 172.21.15.37:0/2436835299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33c06a360 con 0x7fa328003f30 2024-04-26T20:26:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.821+0000 7fa331ffb640 1 -- 172.21.15.37:0/2436835299 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fa33400cc70 con 0x7fa328003f30 2024-04-26T20:26:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.821+0000 7fa331ffb640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa32006cbc0 0x7fa32006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.821+0000 7fa331ffb640 1 -- 172.21.15.37:0/2436835299 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fa33408bb90 con 0x7fa328003f30 2024-04-26T20:26:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.821+0000 7fa340ca5640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa32006cbc0 0x7fa32006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.822+0000 7fa340ca5640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa32006cbc0 0x7fa32006f080 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa32c033000 tx=0x7fa32c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:29.825+0000 7fa331ffb640 1 -- 172.21.15.37:0/2436835299 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa334059f70 con 0x7fa328003f30 2024-04-26T20:26:30.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.013+0000 7fa34370a640 1 -- 172.21.15.37:0/2436835299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa33c14c650 con 0x7fa328003f30 2024-04-26T20:26:30.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.016+0000 7fa331ffb640 1 -- 172.21.15.37:0/2436835299 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fa334059870 con 0x7fa328003f30 2024-04-26T20:26:30.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 -- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa32006cbc0 msgr2=0x7fa32006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:30.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa32006cbc0 0x7fa32006f080 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa32c033000 tx=0x7fa32c008030 comp rx=0 tx=0).stop 2024-04-26T20:26:30.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 -- 172.21.15.37:0/2436835299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 msgr2=0x7fa33c14c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:30.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 0x7fa33c14c110 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa33400e730 tx=0x7fa33400ebf0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 -- 172.21.15.37:0/2436835299 shutdown_connections 2024-04-26T20:26:30.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa32006cbc0 0x7fa32006f080 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa33c0ae9e0 0x7fa33c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa33c15ae40 0x7fa33c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 --2- 172.21.15.37:0/2436835299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa328003f30 0x7fa33c14c110 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 -- 172.21.15.37:0/2436835299 >> 172.21.15.37:0/2436835299 conn(0x7fa33c0d16b0 msgr2=0x7fa33c15f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:30.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 -- 172.21.15.37:0/2436835299 shutdown_connections 2024-04-26T20:26:30.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.018+0000 7fa34370a640 1 -- 172.21.15.37:0/2436835299 wait complete. 2024-04-26T20:26:30.030 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_in_progress_cancelled' 2024-04-26T20:26:30.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.131+0000 7f97e2a62640 1 Processor -- start 2024-04-26T20:26:30.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.131+0000 7f97e2a62640 1 -- start start 2024-04-26T20:26:30.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.131+0000 7f97e2a62640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f97dc159a90 0x7f97dc06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:30.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.131+0000 7f97e2a62640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97dc072e30 con 0x7f97dc070130 2024-04-26T20:26:30.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.131+0000 7f97e2a62640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97dc073110 con 0x7f97dc160320 2024-04-26T20:26:30.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.131+0000 7f97e2a62640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f97dc0733f0 con 0x7f97dc159a90 2024-04-26T20:26:30.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.132+0000 7f97db7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f97dc070130 0x7f97dc15fc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48102/0 (socket says 172.21.15.37:48102) 2024-04-26T20:26:30.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.132+0000 7f97db7fe640 1 -- 172.21.15.37:0/2163124960 learned_addr learned my addr 172.21.15.37:0/2163124960 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:30.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.132+0000 7f97dbfff640 1 --2- 172.21.15.37:0/2163124960 >> v2:172.21.15.134:3300/0 conn(0x7f97dc159a90 0x7f97dc06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:30.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.132+0000 7f97daffd640 1 -- 172.21.15.37:0/2163124960 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3195851742 0 0) 0x7f97dc072e30 con 0x7f97dc070130 2024-04-26T20:26:30.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.132+0000 7f97daffd640 1 -- 172.21.15.37:0/2163124960 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97c4003520 con 0x7f97dc070130 2024-04-26T20:26:30.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.132+0000 7f97dbfff640 1 -- 172.21.15.37:0/2163124960 >> v1:172.21.15.134:6789/0 conn(0x7f97dc160320 legacy=0x7f97dc0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:30.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.132+0000 7f97dbfff640 1 -- 172.21.15.37:0/2163124960 >> v1:172.21.15.37:6789/0 conn(0x7f97dc070130 legacy=0x7f97dc15fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:30.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.132+0000 7f97daffd640 1 -- 172.21.15.37:0/2163124960 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3669843897 0 0) 0x7f97c4003520 con 0x7f97dc070130 2024-04-26T20:26:30.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.132+0000 7f97dbfff640 1 -- 172.21.15.37:0/2163124960 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc0736d0 con 0x7f97dc159a90 2024-04-26T20:26:30.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.133+0000 7f97dbfff640 1 --2- 172.21.15.37:0/2163124960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97dc159a90 0x7f97dc06fbf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0097d0 tx=0x7f97cc02a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f319c42874d738b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:30.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.133+0000 7f97daffd640 1 -- 172.21.15.37:0/2163124960 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc037070 con 0x7f97dc159a90 2024-04-26T20:26:30.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.133+0000 7f97daffd640 1 -- 172.21.15.37:0/2163124960 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97cc02ab60 con 0x7f97dc159a90 2024-04-26T20:26:30.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.133+0000 7f97daffd640 1 -- 172.21.15.37:0/2163124960 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0001670 con 0x7f97dc159a90 2024-04-26T20:26:30.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.134+0000 7f97e2a62640 1 -- 172.21.15.37:0/2163124960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97dc159a90 msgr2=0x7f97dc06fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:30.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.134+0000 7f97e2a62640 1 --2- 172.21.15.37:0/2163124960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97dc159a90 0x7f97dc06fbf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0097d0 tx=0x7f97cc02a0d0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.134+0000 7f97e2a62640 1 -- 172.21.15.37:0/2163124960 shutdown_connections 2024-04-26T20:26:30.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.134+0000 7f97e2a62640 1 --2- 172.21.15.37:0/2163124960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97dc159a90 0x7f97dc06fbf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.134+0000 7f97e2a62640 1 -- 172.21.15.37:0/2163124960 >> 172.21.15.37:0/2163124960 conn(0x7f97dc0d0d50 msgr2=0x7f97dc0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:30.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.134+0000 7f97e2a62640 1 -- 172.21.15.37:0/2163124960 shutdown_connections 2024-04-26T20:26:30.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.134+0000 7f97e2a62640 1 -- 172.21.15.37:0/2163124960 wait complete. 2024-04-26T20:26:30.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.135+0000 7f97e2a62640 1 Processor -- start 2024-04-26T20:26:30.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.135+0000 7f97e2a62640 1 -- start start 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97e2a62640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97dc070130 0x7f97dc15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97e2a62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97dc159a90 0x7f97dc150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97e2a62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97dc160320 0x7f97dc15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97e2a62640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc159ff0 con 0x7f97dc159a90 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97e2a62640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc159e70 con 0x7f97dc160320 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97e2a62640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc0cf0c0 con 0x7f97dc070130 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97dc159a90 0x7f97dc150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97dc159a90 0x7f97dc150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55864/0 (socket says 172.21.15.37:55864) 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97db7fe640 1 -- 172.21.15.37:0/2275647861 learned_addr learned my addr 172.21.15.37:0/2275647861 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97e0fff640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97dc160320 0x7f97dc15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97dbfff640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97dc070130 0x7f97dc15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97db7fe640 1 -- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97dc070130 msgr2=0x7f97dc15ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97db7fe640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97dc070130 0x7f97dc15ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97db7fe640 1 -- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97dc160320 msgr2=0x7f97dc15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:30.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97db7fe640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97dc160320 0x7f97dc15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97db7fe640 1 -- 172.21.15.37:0/2275647861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc15bb00 con 0x7f97dc159a90 2024-04-26T20:26:30.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.136+0000 7f97dbfff640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97dc070130 0x7f97dc15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:30.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.137+0000 7f97db7fe640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97dc159a90 0x7f97dc150820 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f97d00011e0 tx=0x7f97d000c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:30.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.137+0000 7f97d8ff9640 1 -- 172.21.15.37:0/2275647861 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0017070 con 0x7f97dc159a90 2024-04-26T20:26:30.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.137+0000 7f97d8ff9640 1 -- 172.21.15.37:0/2275647861 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d0005ce0 con 0x7f97dc159a90 2024-04-26T20:26:30.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.137+0000 7f97d8ff9640 1 -- 172.21.15.37:0/2275647861 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d00072b0 con 0x7f97dc159a90 2024-04-26T20:26:30.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.137+0000 7f97e2a62640 1 -- 172.21.15.37:0/2275647861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc15b260 con 0x7f97dc159a90 2024-04-26T20:26:30.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.137+0000 7f97e2a62640 1 -- 172.21.15.37:0/2275647861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc15b820 con 0x7f97dc159a90 2024-04-26T20:26:30.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.138+0000 7f97e2a62640 1 -- 172.21.15.37:0/2275647861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97dc06a360 con 0x7f97dc159a90 2024-04-26T20:26:30.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.138+0000 7f97d8ff9640 1 -- 172.21.15.37:0/2275647861 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f97d001a070 con 0x7f97dc159a90 2024-04-26T20:26:30.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.139+0000 7f97d8ff9640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97c806cbc0 0x7f97c806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:30.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.139+0000 7f97d8ff9640 1 -- 172.21.15.37:0/2275647861 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f97d008b040 con 0x7f97dc159a90 2024-04-26T20:26:30.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.139+0000 7f97dbfff640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97c806cbc0 0x7f97c806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:30.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.139+0000 7f97dbfff640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97c806cbc0 0x7f97c806f080 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f97cc034000 tx=0x7f97cc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:30.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.143+0000 7f97d8ff9640 1 -- 172.21.15.37:0/2275647861 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f97d0058d90 con 0x7f97dc159a90 2024-04-26T20:26:30.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.400+0000 7f97e2a62640 1 -- 172.21.15.37:0/2275647861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_in_progress_cancelled"]} v 0) -- 0x7f97dc0d0730 con 0x7f97dc159a90 2024-04-26T20:26:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.687+0000 7f97d8ff9640 1 -- 172.21.15.37:0/2275647861 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_in_progress_cancelled"]}]=0 v224) ==== 180+0+0 (secure 0 0 0) 0x7f97d005d090 con 0x7f97dc159a90 2024-04-26T20:26:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.689+0000 7f97e2a62640 1 -- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97c806cbc0 msgr2=0x7f97c806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.690+0000 7f97e2a62640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97c806cbc0 0x7f97c806f080 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f97cc034000 tx=0x7f97cc02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.690+0000 7f97e2a62640 1 -- 172.21.15.37:0/2275647861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97dc159a90 msgr2=0x7f97dc150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.690+0000 7f97e2a62640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97dc159a90 0x7f97dc150820 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f97d00011e0 tx=0x7f97d000c9d0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.690+0000 7f97e2a62640 1 -- 172.21.15.37:0/2275647861 shutdown_connections 2024-04-26T20:26:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.690+0000 7f97e2a62640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97dc160320 0x7f97dc15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.690+0000 7f97e2a62640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97dc159a90 0x7f97dc150820 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.690+0000 7f97e2a62640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f97c806cbc0 0x7f97c806f080 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.690+0000 7f97e2a62640 1 --2- 172.21.15.37:0/2275647861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97dc070130 0x7f97dc15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.690+0000 7f97e2a62640 1 -- 172.21.15.37:0/2275647861 >> 172.21.15.37:0/2275647861 conn(0x7f97dc0d0d50 msgr2=0x7f97dc157ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.690+0000 7f97e2a62640 1 -- 172.21.15.37:0/2275647861 shutdown_connections 2024-04-26T20:26:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:30.690+0000 7f97e2a62640 1 -- 172.21.15.37:0/2275647861 wait complete. 2024-04-26T20:26:30.706 INFO:tasks.cephfs_test_runner:ensure failure status is shown when clone is cancelled during in-progress state and validate the reason ... ERROR 2024-04-26T20:26:30.706 INFO:tasks.cephfs_test_runner: 2024-04-26T20:26:30.706 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-26T20:26:30.706 INFO:tasks.cephfs_test_runner:ERROR: test_clone_failure_status_in_progress_cancelled (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2024-04-26T20:26:30.706 INFO:tasks.cephfs_test_runner:ensure failure status is shown when clone is cancelled during in-progress state and validate the reason 2024-04-26T20:26:30.706 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T20:26:30.706 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 6690, in test_clone_failure_status_in_progress_cancelled 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: self._fs_cmd("subvolume", "create", self.volname, subvolume, "--mode=777") 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 33, in _fs_cmd 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue().strip() 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-26T20:26:30.707 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi037 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol76120 --mode=777' 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner: 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner:Ran 2 tests in 253.072s 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner: 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner: 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner:ERROR: test_clone_failure_status_in_progress_cancelled (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner:ensure failure status is shown when clone is cancelled during in-progress state and validate the reason 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 6690, in test_clone_failure_status_in_progress_cancelled 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner: self._fs_cmd("subvolume", "create", self.volname, subvolume, "--mode=777") 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 33, in _fs_cmd 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue().strip() 2024-04-26T20:26:30.708 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi037 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol76120 --mode=777' 2024-04-26T20:26:30.709 INFO:tasks.cephfs_test_runner: 2024-04-26T20:26:30.710 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_clone_failure_status_in_progress_cancelled (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2024-04-26T20:26:30.943 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=e3e69736361a443b8dd71e3500ecf9d0 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_clone_failure_status_in_progress_cancelled (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2024-04-26T20:26:30.944 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T20:26:30.954 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T20:26:30.984 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T20:26:30.984 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:26:30.984 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:31.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=371 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:26:31.004 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T20:26:31.004 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:31.060 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=371 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:26:31.060 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:26:31.060 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:31.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17743 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:26:31.079 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-26T20:26:31.079 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:31.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17743 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:26:31.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.136 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T20:26:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:26:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:26:31.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.152 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:26:31.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.218 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:26:31.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-26T20:26:31.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.370 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T20:26:31.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.386 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-26T20:26:31.453 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-04-26T20:26:31.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:26:31.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.455 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-26T20:26:31.473 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T20:26:31.473 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:26:31.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.474 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:26:31.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.541 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:26:31.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.608 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-04-26T20:26:31.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.705 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-26T20:26:31.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:31.722 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-04-26T20:26:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:Cannot find device "ceph-brx" 2024-04-26T20:26:31.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:26:31.791 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T20:26:31.799 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T20:26:31.799 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:26:31.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.902+0000 7ffbacbe7640 1 Processor -- start 2024-04-26T20:26:31.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.902+0000 7ffbacbe7640 1 -- start start 2024-04-26T20:26:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.902+0000 7ffbacbe7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8159a90 0x7ffba806fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffbacbe7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffba8160320 0x7ffba80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffbacbe7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba8072e30 con 0x7ffba8070130 2024-04-26T20:26:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffbacbe7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ffba8073110 con 0x7ffba8160320 2024-04-26T20:26:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffbacbe7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffba80733f0 con 0x7ffba8159a90 2024-04-26T20:26:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffba6d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffba8160320 0x7ffba80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffba659c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8159a90 0x7ffba806fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffba659c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8159a90 0x7ffba806fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.37:40540/0 (socket says 172.21.15.37:40540) 2024-04-26T20:26:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffba6d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffba8160320 0x7ffba80ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.37:43878/0 (socket says 172.21.15.37:43878) 2024-04-26T20:26:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffba659c640 1 -- 172.21.15.37:0/3994449586 learned_addr learned my addr 172.21.15.37:0/3994449586 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:31.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffba659c640 1 -- 172.21.15.37:0/3994449586 >> v2:172.21.15.134:3300/0 conn(0x7ffba8160320 msgr2=0x7ffba80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:31.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffba659c640 1 --2- 172.21.15.37:0/3994449586 >> v2:172.21.15.134:3300/0 conn(0x7ffba8160320 0x7ffba80ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffba659c640 1 -- 172.21.15.37:0/3994449586 >> v1:172.21.15.134:6789/0 conn(0x7ffba8070130 legacy=0x7ffba815fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffba659c640 1 -- 172.21.15.37:0/3994449586 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba80736d0 con 0x7ffba8159a90 2024-04-26T20:26:31.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.903+0000 7ffba6d9d640 1 --2- 172.21.15.37:0/3994449586 >> v2:172.21.15.134:3300/0 conn(0x7ffba8160320 0x7ffba80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.904+0000 7ffba659c640 1 --2- 172.21.15.37:0/3994449586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8159a90 0x7ffba806fbf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009870 tx=0x7ffb9802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346765168e9e750e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.904+0000 7ffba559a640 1 -- 172.21.15.37:0/3994449586 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98037070 con 0x7ffba8159a90 2024-04-26T20:26:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.904+0000 7ffba559a640 1 -- 172.21.15.37:0/3994449586 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb98032440 con 0x7ffba8159a90 2024-04-26T20:26:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.904+0000 7ffba559a640 1 -- 172.21.15.37:0/3994449586 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98032810 con 0x7ffba8159a90 2024-04-26T20:26:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.905+0000 7ffbacbe7640 1 -- 172.21.15.37:0/3994449586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8159a90 msgr2=0x7ffba806fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.905+0000 7ffbacbe7640 1 --2- 172.21.15.37:0/3994449586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8159a90 0x7ffba806fbf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009870 tx=0x7ffb9802a130 comp rx=0 tx=0).stop 2024-04-26T20:26:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.905+0000 7ffbacbe7640 1 -- 172.21.15.37:0/3994449586 shutdown_connections 2024-04-26T20:26:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.905+0000 7ffbacbe7640 1 --2- 172.21.15.37:0/3994449586 >> v2:172.21.15.134:3300/0 conn(0x7ffba8160320 0x7ffba80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.905+0000 7ffbacbe7640 1 --2- 172.21.15.37:0/3994449586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8159a90 0x7ffba806fbf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.905+0000 7ffbacbe7640 1 -- 172.21.15.37:0/3994449586 >> 172.21.15.37:0/3994449586 conn(0x7ffba80d0d50 msgr2=0x7ffba80d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.905+0000 7ffbacbe7640 1 -- 172.21.15.37:0/3994449586 shutdown_connections 2024-04-26T20:26:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.905+0000 7ffbacbe7640 1 -- 172.21.15.37:0/3994449586 wait complete. 2024-04-26T20:26:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.906+0000 7ffbacbe7640 1 Processor -- start 2024-04-26T20:26:31.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.906+0000 7ffbacbe7640 1 -- start start 2024-04-26T20:26:31.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffbacbe7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba8070130 0x7ffba8141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffbacbe7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8159a90 0x7ffba80b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffbacbe7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8160320 0x7ffba80b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffbacbe7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffba8159ff0 con 0x7ffba8159a90 2024-04-26T20:26:31.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffbacbe7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffba8159e70 con 0x7ffba8070130 2024-04-26T20:26:31.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffbacbe7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffba80ceef0 con 0x7ffba8160320 2024-04-26T20:26:31.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffba5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8159a90 0x7ffba80b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffba5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8159a90 0x7ffba80b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55876/0 (socket says 172.21.15.37:55876) 2024-04-26T20:26:31.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffba6d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8160320 0x7ffba80b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffba5d9b640 1 -- 172.21.15.37:0/331853057 learned_addr learned my addr 172.21.15.37:0/331853057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:31.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffba659c640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba8070130 0x7ffba8141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffba5d9b640 1 -- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8160320 msgr2=0x7ffba80b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffba5d9b640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8160320 0x7ffba80b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffba5d9b640 1 -- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba8070130 msgr2=0x7ffba8141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffba5d9b640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba8070130 0x7ffba8141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.907+0000 7ffba5d9b640 1 -- 172.21.15.37:0/331853057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba8153470 con 0x7ffba8159a90 2024-04-26T20:26:31.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.908+0000 7ffba5d9b640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8159a90 0x7ffba80b6050 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00c400 tx=0x7ffb9c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.908+0000 7ffb96ffd640 1 -- 172.21.15.37:0/331853057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0052d0 con 0x7ffba8159a90 2024-04-26T20:26:31.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.908+0000 7ffb96ffd640 1 -- 172.21.15.37:0/331853057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c005490 con 0x7ffba8159a90 2024-04-26T20:26:31.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.908+0000 7ffbacbe7640 1 -- 172.21.15.37:0/331853057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba8153750 con 0x7ffba8159a90 2024-04-26T20:26:31.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.908+0000 7ffb96ffd640 1 -- 172.21.15.37:0/331853057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c007260 con 0x7ffba8159a90 2024-04-26T20:26:31.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.908+0000 7ffbacbe7640 1 -- 172.21.15.37:0/331853057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba8141b70 con 0x7ffba8159a90 2024-04-26T20:26:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.909+0000 7ffbacbe7640 1 -- 172.21.15.37:0/331853057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba806a360 con 0x7ffba8159a90 2024-04-26T20:26:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.909+0000 7ffb96ffd640 1 -- 172.21.15.37:0/331853057 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7ffb9c007400 con 0x7ffba8159a90 2024-04-26T20:26:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.909+0000 7ffb96ffd640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb8806cbc0 0x7ffb8806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.910+0000 7ffb96ffd640 1 -- 172.21.15.37:0/331853057 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7ffb9c08b090 con 0x7ffba8159a90 2024-04-26T20:26:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.910+0000 7ffba659c640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb8806cbc0 0x7ffb8806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.910+0000 7ffba659c640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb8806cbc0 0x7ffb8806f080 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffb98034000 tx=0x7ffb9802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:31.914+0000 7ffb96ffd640 1 -- 172.21.15.37:0/331853057 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb9c058e20 con 0x7ffba8159a90 2024-04-26T20:26:32.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.097+0000 7ffbacbe7640 1 -- 172.21.15.37:0/331853057 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffba80cec40 con 0x7ffb8806cbc0 2024-04-26T20:26:32.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.099+0000 7ffb96ffd640 1 -- 172.21.15.37:0/331853057 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13426 (secure 0 0 0) 0x7ffba80cec40 con 0x7ffb8806cbc0 2024-04-26T20:26:32.099 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:32.100 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:26:32.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.100+0000 7ffbacbe7640 1 -- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb8806cbc0 msgr2=0x7ffb8806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.100+0000 7ffbacbe7640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb8806cbc0 0x7ffb8806f080 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffb98034000 tx=0x7ffb9802e000 comp rx=0 tx=0).stop 2024-04-26T20:26:32.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.100+0000 7ffbacbe7640 1 -- 172.21.15.37:0/331853057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8159a90 msgr2=0x7ffba80b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.101+0000 7ffbacbe7640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8159a90 0x7ffba80b6050 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00c400 tx=0x7ffb9c00c800 comp rx=0 tx=0).stop 2024-04-26T20:26:32.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.101+0000 7ffbacbe7640 1 -- 172.21.15.37:0/331853057 shutdown_connections 2024-04-26T20:26:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.101+0000 7ffbacbe7640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ffb8806cbc0 0x7ffb8806f080 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.101+0000 7ffbacbe7640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffba8160320 0x7ffba80b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.101+0000 7ffbacbe7640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8159a90 0x7ffba80b6050 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.101+0000 7ffbacbe7640 1 --2- 172.21.15.37:0/331853057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba8070130 0x7ffba8141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.101+0000 7ffbacbe7640 1 -- 172.21.15.37:0/331853057 >> 172.21.15.37:0/331853057 conn(0x7ffba80d0d50 msgr2=0x7ffba8073f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.101+0000 7ffbacbe7640 1 -- 172.21.15.37:0/331853057 shutdown_connections 2024-04-26T20:26:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.101+0000 7ffbacbe7640 1 -- 172.21.15.37:0/331853057 wait complete. 2024-04-26T20:26:32.112 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-04-26T20:26:31.403912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234308,"kb_used_data":5676,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754740412,"statfs":{"total":773094113280,"available":772854181888,"internally_reserved":0,"allocated":5812224,"data_stored":3080855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4050,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001388"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":224,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T20:24:32.939772+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:24:32.939772+0000","last_peered":"2024-04-26T20:24:32.939772+0000","last_clean":"2024-04-26T20:24:32.939772+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:24:32.939772+0000","last_undegraded":"2024-04-26T20:24:32.939772+0000","last_fullsized":"2024-04-26T20:24:32.939772+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1163264,"data_stored":824623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":234837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":240374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":577536,"data_stored":237558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":577536,"data_stored":234803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":243719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":581632,"data_stored":240374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1163264,"data_stored":824567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:26:32.113 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:26:32.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.212+0000 7f3e80ac8640 1 Processor -- start 2024-04-26T20:26:32.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.212+0000 7f3e80ac8640 1 -- start start 2024-04-26T20:26:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.213+0000 7f3e80ac8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3e7c162b40 0x7f3e7c0d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.213+0000 7f3e80ac8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e7c0d3c80 0x7f3e7c0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.213+0000 7f3e80ac8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7c0d0210 con 0x7f3e7c0d3820 2024-04-26T20:26:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.213+0000 7f3e80ac8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3e7c0d04f0 con 0x7f3e7c162b40 2024-04-26T20:26:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.213+0000 7f3e80ac8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7c0d07d0 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.213+0000 7f3e7a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3e7c162b40 0x7f3e7c0d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.213+0000 7f3e79d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3e7c0d3820 0x7f3e7c0d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48112/0 (socket says 172.21.15.37:48112) 2024-04-26T20:26:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.213+0000 7f3e79d9b640 1 -- 172.21.15.37:0/1140476496 learned_addr learned my addr 172.21.15.37:0/1140476496 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.214+0000 7f3e7ad9d640 1 --2- 172.21.15.37:0/1140476496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e7c0d3c80 0x7f3e7c0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.214+0000 7f3e7a59c640 1 -- 172.21.15.37:0/1140476496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e7c0d3c80 msgr2=0x7f3e7c0d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.214+0000 7f3e7a59c640 1 --2- 172.21.15.37:0/1140476496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e7c0d3c80 0x7f3e7c0d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.214+0000 7f3e7a59c640 1 -- 172.21.15.37:0/1140476496 >> v1:172.21.15.37:6789/0 conn(0x7f3e7c0d3820 legacy=0x7f3e7c0d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.214+0000 7f3e7959a640 1 -- 172.21.15.37:0/1140476496 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917418585 0 0) 0x7f3e7c0d0210 con 0x7f3e7c0d3820 2024-04-26T20:26:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.214+0000 7f3e7a59c640 1 -- 172.21.15.37:0/1140476496 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c0d0ab0 con 0x7f3e7c162b40 2024-04-26T20:26:32.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.214+0000 7f3e7a59c640 1 --2- 172.21.15.37:0/1140476496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c162b40 0x7f3e7c0d5bd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009d90 tx=0x7f3e6c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7beaeae1084418d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.215+0000 7f3e7959a640 1 -- 172.21.15.37:0/1140476496 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c007740 con 0x7f3e7c162b40 2024-04-26T20:26:32.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.215+0000 7f3e7959a640 1 -- 172.21.15.37:0/1140476496 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e6c0078e0 con 0x7f3e7c162b40 2024-04-26T20:26:32.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.215+0000 7f3e7959a640 1 -- 172.21.15.37:0/1140476496 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c007cb0 con 0x7f3e7c162b40 2024-04-26T20:26:32.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.215+0000 7f3e80ac8640 1 -- 172.21.15.37:0/1140476496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c162b40 msgr2=0x7f3e7c0d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.215+0000 7f3e80ac8640 1 --2- 172.21.15.37:0/1140476496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c162b40 0x7f3e7c0d5bd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009d90 tx=0x7f3e6c02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.215+0000 7f3e80ac8640 1 -- 172.21.15.37:0/1140476496 shutdown_connections 2024-04-26T20:26:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.215+0000 7f3e80ac8640 1 --2- 172.21.15.37:0/1140476496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e7c0d3c80 0x7f3e7c0d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.215+0000 7f3e80ac8640 1 --2- 172.21.15.37:0/1140476496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c162b40 0x7f3e7c0d5bd0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.215+0000 7f3e80ac8640 1 -- 172.21.15.37:0/1140476496 >> 172.21.15.37:0/1140476496 conn(0x7f3e7c09e550 msgr2=0x7f3e7c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.215+0000 7f3e80ac8640 1 -- 172.21.15.37:0/1140476496 shutdown_connections 2024-04-26T20:26:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.215+0000 7f3e80ac8640 1 -- 172.21.15.37:0/1140476496 wait complete. 2024-04-26T20:26:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.217+0000 7f3e80ac8640 1 Processor -- start 2024-04-26T20:26:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.217+0000 7f3e80ac8640 1 -- start start 2024-04-26T20:26:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.217+0000 7f3e80ac8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e7c0d3820 0x7f3e7c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.217+0000 7f3e80ac8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c0d3c80 0x7f3e7c152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.217+0000 7f3e80ac8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e7c162b40 0x7f3e7c160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.217+0000 7f3e80ac8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c0b5bb0 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.217+0000 7f3e80ac8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c0d4110 con 0x7f3e7c0d3820 2024-04-26T20:26:32.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.217+0000 7f3e80ac8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7c0d6110 con 0x7f3e7c162b40 2024-04-26T20:26:32.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.217+0000 7f3e79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c0d3c80 0x7f3e7c152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.217+0000 7f3e79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c0d3c80 0x7f3e7c152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55890/0 (socket says 172.21.15.37:55890) 2024-04-26T20:26:32.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.217+0000 7f3e79d9b640 1 -- 172.21.15.37:0/3136050270 learned_addr learned my addr 172.21.15.37:0/3136050270 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:32.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e7a59c640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e7c0d3820 0x7f3e7c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e7ad9d640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e7c162b40 0x7f3e7c160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e79d9b640 1 -- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e7c162b40 msgr2=0x7f3e7c160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e79d9b640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e7c162b40 0x7f3e7c160f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e79d9b640 1 -- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e7c0d3820 msgr2=0x7f3e7c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e79d9b640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e7c0d3820 0x7f3e7c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e79d9b640 1 -- 172.21.15.37:0/3136050270 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c0b9160 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e79d9b640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c0d3c80 0x7f3e7c152ae0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c430 tx=0x7f3e6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e72ffd640 1 -- 172.21.15.37:0/3136050270 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e640074e0 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e72ffd640 1 -- 172.21.15.37:0/3136050270 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e640047e0 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e80ac8640 1 -- 172.21.15.37:0/3136050270 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7c0b94a0 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e72ffd640 1 -- 172.21.15.37:0/3136050270 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64004bd0 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.218+0000 7f3e80ac8640 1 -- 172.21.15.37:0/3136050270 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e7c0b9ce0 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.219+0000 7f3e80ac8640 1 -- 172.21.15.37:0/3136050270 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7c0d0210 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.220+0000 7f3e72ffd640 1 -- 172.21.15.37:0/3136050270 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f3e64011070 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.220+0000 7f3e72ffd640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e5c06cbc0 0x7f3e5c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.220+0000 7f3e72ffd640 1 -- 172.21.15.37:0/3136050270 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f3e6408b770 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.220+0000 7f3e7a59c640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e5c06cbc0 0x7f3e5c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.221+0000 7f3e7a59c640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e5c06cbc0 0x7f3e5c06f080 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c034000 tx=0x7f3e6c02f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.224+0000 7f3e72ffd640 1 -- 172.21.15.37:0/3136050270 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e7c0d0210 con 0x7f3e7c0d3c80 2024-04-26T20:26:32.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.407+0000 7f3e80ac8640 1 -- 172.21.15.37:0/3136050270 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e7c0b8eb0 con 0x7f3e5c06cbc0 2024-04-26T20:26:32.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.408+0000 7f3e72ffd640 1 -- 172.21.15.37:0/3136050270 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13426 (secure 0 0 0) 0x7f3e7c0b8eb0 con 0x7f3e5c06cbc0 2024-04-26T20:26:32.409 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:32.410 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:26:32.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 -- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e5c06cbc0 msgr2=0x7f3e5c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e5c06cbc0 0x7f3e5c06f080 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c034000 tx=0x7f3e6c02f000 comp rx=0 tx=0).stop 2024-04-26T20:26:32.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 -- 172.21.15.37:0/3136050270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c0d3c80 msgr2=0x7f3e7c152ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c0d3c80 0x7f3e7c152ae0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c430 tx=0x7f3e6400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 -- 172.21.15.37:0/3136050270 shutdown_connections 2024-04-26T20:26:32.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3e5c06cbc0 0x7f3e5c06f080 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e7c162b40 0x7f3e7c160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c0d3c80 0x7f3e7c152ae0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 --2- 172.21.15.37:0/3136050270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e7c0d3820 0x7f3e7c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 -- 172.21.15.37:0/3136050270 >> 172.21.15.37:0/3136050270 conn(0x7f3e7c09e550 msgr2=0x7f3e7c06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 -- 172.21.15.37:0/3136050270 shutdown_connections 2024-04-26T20:26:32.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.411+0000 7f3e80ac8640 1 -- 172.21.15.37:0/3136050270 wait complete. 2024-04-26T20:26:32.422 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-04-26T20:26:31.403912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234308,"kb_used_data":5676,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754740412,"statfs":{"total":773094113280,"available":772854181888,"internally_reserved":0,"allocated":5812224,"data_stored":3080855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4050,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001388"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":224,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T20:24:32.939772+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:24:32.939772+0000","last_peered":"2024-04-26T20:24:32.939772+0000","last_clean":"2024-04-26T20:24:32.939772+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:24:32.939772+0000","last_undegraded":"2024-04-26T20:24:32.939772+0000","last_fullsized":"2024-04-26T20:24:32.939772+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1163264,"data_stored":824623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":234837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":240374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":577536,"data_stored":237558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":577536,"data_stored":234803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":243719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":581632,"data_stored":240374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1163264,"data_stored":824567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:26:32.423 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T20:26:32.423 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:26:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.523+0000 7f040c389640 1 Processor -- start 2024-04-26T20:26:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.523+0000 7f040c389640 1 -- start start 2024-04-26T20:26:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f040c389640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f040406ea40 0x7f04040d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f040c389640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f04040cf830 0x7f04040cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f040c389640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04040d0180 con 0x7f0404162e40 2024-04-26T20:26:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f040c389640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f0404163220 con 0x7f040406ea40 2024-04-26T20:26:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f040c389640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f04040d0eb0 con 0x7f04040cf830 2024-04-26T20:26:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f0409924640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f040406ea40 0x7f04040d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f0409924640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f040406ea40 0x7f04040d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55904/0 (socket says 172.21.15.37:55904) 2024-04-26T20:26:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f0409924640 1 -- 172.21.15.37:0/1542105792 learned_addr learned my addr 172.21.15.37:0/1542105792 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f040a926640 1 --2- 172.21.15.37:0/1542105792 >> v2:172.21.15.134:3300/0 conn(0x7f04040cf830 0x7f04040cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f0409924640 1 -- 172.21.15.37:0/1542105792 >> v2:172.21.15.134:3300/0 conn(0x7f04040cf830 msgr2=0x7f04040cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f0409924640 1 --2- 172.21.15.37:0/1542105792 >> v2:172.21.15.134:3300/0 conn(0x7f04040cf830 0x7f04040cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f0409924640 1 -- 172.21.15.37:0/1542105792 >> v1:172.21.15.134:6789/0 conn(0x7f0404162e40 legacy=0x7f04040b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f0409924640 1 -- 172.21.15.37:0/1542105792 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04040b1d70 con 0x7f040406ea40 2024-04-26T20:26:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.524+0000 7f040a926640 1 --2- 172.21.15.37:0/1542105792 >> v2:172.21.15.134:3300/0 conn(0x7f04040cf830 0x7f04040cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.525+0000 7f0409924640 1 --2- 172.21.15.37:0/1542105792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f040406ea40 0x7f04040d1340 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f04040737a0 tx=0x7f03f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba04dbbf41dd4fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.525+0000 7f0409123640 1 -- 172.21.15.37:0/1542105792 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4037070 con 0x7f040406ea40 2024-04-26T20:26:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.525+0000 7f0409123640 1 -- 172.21.15.37:0/1542105792 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4032440 con 0x7f040406ea40 2024-04-26T20:26:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.525+0000 7f0409123640 1 -- 172.21.15.37:0/1542105792 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4032810 con 0x7f040406ea40 2024-04-26T20:26:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.526+0000 7f040c389640 1 -- 172.21.15.37:0/1542105792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f040406ea40 msgr2=0x7f04040d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.526+0000 7f040c389640 1 --2- 172.21.15.37:0/1542105792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f040406ea40 0x7f04040d1340 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f04040737a0 tx=0x7f03f402a1a0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.526+0000 7f040c389640 1 -- 172.21.15.37:0/1542105792 shutdown_connections 2024-04-26T20:26:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.526+0000 7f040c389640 1 --2- 172.21.15.37:0/1542105792 >> v2:172.21.15.134:3300/0 conn(0x7f04040cf830 0x7f04040cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.526+0000 7f040c389640 1 --2- 172.21.15.37:0/1542105792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f040406ea40 0x7f04040d1340 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.526+0000 7f040c389640 1 -- 172.21.15.37:0/1542105792 >> 172.21.15.37:0/1542105792 conn(0x7f040409ba50 msgr2=0x7f040409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.526+0000 7f040c389640 1 -- 172.21.15.37:0/1542105792 shutdown_connections 2024-04-26T20:26:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.526+0000 7f040c389640 1 -- 172.21.15.37:0/1542105792 wait complete. 2024-04-26T20:26:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.527+0000 7f040c389640 1 Processor -- start 2024-04-26T20:26:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.527+0000 7f040c389640 1 -- start start 2024-04-26T20:26:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f040c389640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f040406ea40 0x7f040415c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f040c389640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04040cf830 0x7f04040b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f040c389640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0404162e40 0x7f040415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f040c389640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04040d1f20 con 0x7f04040cf830 2024-04-26T20:26:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f040c389640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04040d1c40 con 0x7f040406ea40 2024-04-26T20:26:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f040c389640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f04040ce010 con 0x7f0404162e40 2024-04-26T20:26:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f0409924640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04040cf830 0x7f04040b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f0409924640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04040cf830 0x7f04040b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55906/0 (socket says 172.21.15.37:55906) 2024-04-26T20:26:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f0409924640 1 -- 172.21.15.37:0/2817018057 learned_addr learned my addr 172.21.15.37:0/2817018057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f040a125640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f040406ea40 0x7f040415c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f040a926640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0404162e40 0x7f040415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f0409924640 1 -- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0404162e40 msgr2=0x7f040415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f0409924640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0404162e40 0x7f040415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f0409924640 1 -- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f040406ea40 msgr2=0x7f040415c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f0409924640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f040406ea40 0x7f040415c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f0409924640 1 -- 172.21.15.37:0/2817018057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04040ca220 con 0x7f04040cf830 2024-04-26T20:26:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.528+0000 7f040a926640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0404162e40 0x7f040415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.529+0000 7f0409924640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04040cf830 0x7f04040b31a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f03f4000c00 tx=0x7f03f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.529+0000 7f03faffd640 1 -- 172.21.15.37:0/2817018057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4037040 con 0x7f04040cf830 2024-04-26T20:26:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.529+0000 7f03faffd640 1 -- 172.21.15.37:0/2817018057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4042070 con 0x7f04040cf830 2024-04-26T20:26:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.529+0000 7f040c389640 1 -- 172.21.15.37:0/2817018057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04040ca500 con 0x7f04040cf830 2024-04-26T20:26:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.529+0000 7f03faffd640 1 -- 172.21.15.37:0/2817018057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4030b90 con 0x7f04040cf830 2024-04-26T20:26:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.529+0000 7f040c389640 1 -- 172.21.15.37:0/2817018057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04040caac0 con 0x7f04040cf830 2024-04-26T20:26:32.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.530+0000 7f040c389640 1 -- 172.21.15.37:0/2817018057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04040cdd50 con 0x7f04040cf830 2024-04-26T20:26:32.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.530+0000 7f03faffd640 1 -- 172.21.15.37:0/2817018057 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f03f4047460 con 0x7f04040cf830 2024-04-26T20:26:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.531+0000 7f03faffd640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03e806cbc0 0x7f03e806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.531+0000 7f03faffd640 1 -- 172.21.15.37:0/2817018057 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f03f40ac960 con 0x7f04040cf830 2024-04-26T20:26:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.531+0000 7f040a125640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03e806cbc0 0x7f03e806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.531+0000 7f040a125640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03e806cbc0 0x7f03e806f080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0400006fd0 tx=0x7f0400009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.535+0000 7f03faffd640 1 -- 172.21.15.37:0/2817018057 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f04040d0180 con 0x7f04040cf830 2024-04-26T20:26:32.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.718+0000 7f040c389640 1 -- 172.21.15.37:0/2817018057 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f040406a360 con 0x7f03e806cbc0 2024-04-26T20:26:32.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.719+0000 7f03faffd640 1 -- 172.21.15.37:0/2817018057 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13426 (secure 0 0 0) 0x7f040406a360 con 0x7f03e806cbc0 2024-04-26T20:26:32.721 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:32.721 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:26:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.721+0000 7f040c389640 1 -- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03e806cbc0 msgr2=0x7f03e806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.721+0000 7f040c389640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03e806cbc0 0x7f03e806f080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0400006fd0 tx=0x7f0400009060 comp rx=0 tx=0).stop 2024-04-26T20:26:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.721+0000 7f040c389640 1 -- 172.21.15.37:0/2817018057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04040cf830 msgr2=0x7f04040b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.721+0000 7f040c389640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04040cf830 0x7f04040b31a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f03f4000c00 tx=0x7f03f40046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.721+0000 7f040c389640 1 -- 172.21.15.37:0/2817018057 shutdown_connections 2024-04-26T20:26:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.721+0000 7f040c389640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03e806cbc0 0x7f03e806f080 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.721+0000 7f040c389640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0404162e40 0x7f040415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.721+0000 7f040c389640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04040cf830 0x7f04040b31a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.722+0000 7f040c389640 1 --2- 172.21.15.37:0/2817018057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f040406ea40 0x7f040415c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.722+0000 7f040c389640 1 -- 172.21.15.37:0/2817018057 >> 172.21.15.37:0/2817018057 conn(0x7f040409ba50 msgr2=0x7f040415d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.722+0000 7f040c389640 1 -- 172.21.15.37:0/2817018057 shutdown_connections 2024-04-26T20:26:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.722+0000 7f040c389640 1 -- 172.21.15.37:0/2817018057 wait complete. 2024-04-26T20:26:32.733 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-04-26T20:26:31.403912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234308,"kb_used_data":5676,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754740412,"statfs":{"total":773094113280,"available":772854181888,"internally_reserved":0,"allocated":5812224,"data_stored":3080855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4050,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001388"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":224,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T20:24:32.939772+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:24:32.939772+0000","last_peered":"2024-04-26T20:24:32.939772+0000","last_clean":"2024-04-26T20:24:32.939772+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:24:32.939772+0000","last_undegraded":"2024-04-26T20:24:32.939772+0000","last_fullsized":"2024-04-26T20:24:32.939772+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1163264,"data_stored":824623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":234837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":240374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":577536,"data_stored":237558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":577536,"data_stored":234803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":243719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":581632,"data_stored":240374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1163264,"data_stored":824567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:26:32.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:32.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.833+0000 7f45cf028640 1 Processor -- start 2024-04-26T20:26:32.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.833+0000 7f45cf028640 1 -- start start 2024-04-26T20:26:32.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45cf028640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f45c8157800 0x7f45c815e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45cf028640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f45c80cefa0 0x7f45c8141a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45cf028640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c80d0770 con 0x7f45c8073180 2024-04-26T20:26:32.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45cf028640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f45c806c590 con 0x7f45c8157800 2024-04-26T20:26:32.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45cf028640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f45c80d1c40 con 0x7f45c80cefa0 2024-04-26T20:26:32.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45ccdc4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f45c8157800 0x7f45c815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45ccdc4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f45c8157800 0x7f45c815e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55914/0 (socket says 172.21.15.37:55914) 2024-04-26T20:26:32.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45ccdc4640 1 -- 172.21.15.37:0/4099335662 learned_addr learned my addr 172.21.15.37:0/4099335662 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:32.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45cd5c5640 1 --2- 172.21.15.37:0/4099335662 >> v2:172.21.15.134:3300/0 conn(0x7f45c80cefa0 0x7f45c8141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45ccdc4640 1 -- 172.21.15.37:0/4099335662 >> v2:172.21.15.134:3300/0 conn(0x7f45c80cefa0 msgr2=0x7f45c8141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:32.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45ccdc4640 1 --2- 172.21.15.37:0/4099335662 >> v2:172.21.15.134:3300/0 conn(0x7f45c80cefa0 0x7f45c8141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45ccdc4640 1 -- 172.21.15.37:0/4099335662 >> v1:172.21.15.134:6789/0 conn(0x7f45c8073180 legacy=0x7f45c80ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:32.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45ccdc4640 1 -- 172.21.15.37:0/4099335662 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c815e5b0 con 0x7f45c8157800 2024-04-26T20:26:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.834+0000 7f45cd5c5640 1 --2- 172.21.15.37:0/4099335662 >> v2:172.21.15.134:3300/0 conn(0x7f45c80cefa0 0x7f45c8141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.835+0000 7f45ccdc4640 1 --2- 172.21.15.37:0/4099335662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45c8157800 0x7f45c815e070 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f45c80b4980 tx=0x7f45c002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4216537c899a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.835+0000 7f45c77fe640 1 -- 172.21.15.37:0/4099335662 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0036070 con 0x7f45c8157800 2024-04-26T20:26:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.835+0000 7f45c77fe640 1 -- 172.21.15.37:0/4099335662 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c0031600 con 0x7f45c8157800 2024-04-26T20:26:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.835+0000 7f45c77fe640 1 -- 172.21.15.37:0/4099335662 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c00319d0 con 0x7f45c8157800 2024-04-26T20:26:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.836+0000 7f45cf028640 1 -- 172.21.15.37:0/4099335662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45c8157800 msgr2=0x7f45c815e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.836+0000 7f45cf028640 1 --2- 172.21.15.37:0/4099335662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45c8157800 0x7f45c815e070 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f45c80b4980 tx=0x7f45c002a6f0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.836+0000 7f45cf028640 1 -- 172.21.15.37:0/4099335662 shutdown_connections 2024-04-26T20:26:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.836+0000 7f45cf028640 1 --2- 172.21.15.37:0/4099335662 >> v2:172.21.15.134:3300/0 conn(0x7f45c80cefa0 0x7f45c8141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.836+0000 7f45cf028640 1 --2- 172.21.15.37:0/4099335662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45c8157800 0x7f45c815e070 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.836+0000 7f45cf028640 1 -- 172.21.15.37:0/4099335662 >> 172.21.15.37:0/4099335662 conn(0x7f45c80b3b70 msgr2=0x7f45c80b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.836+0000 7f45cf028640 1 -- 172.21.15.37:0/4099335662 shutdown_connections 2024-04-26T20:26:32.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.836+0000 7f45cf028640 1 -- 172.21.15.37:0/4099335662 wait complete. 2024-04-26T20:26:32.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.837+0000 7f45cf028640 1 Processor -- start 2024-04-26T20:26:32.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.837+0000 7f45cf028640 1 -- start start 2024-04-26T20:26:32.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cf028640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45c8073180 0x7f45c81357a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cf028640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c80cefa0 0x7f45c81585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cf028640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45c8157800 0x7f45c815f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cf028640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8073c00 con 0x7f45c8157800 2024-04-26T20:26:32.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cf028640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8073920 con 0x7f45c80cefa0 2024-04-26T20:26:32.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cf028640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f45c8073ee0 con 0x7f45c8073180 2024-04-26T20:26:32.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cd5c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45c8157800 0x7f45c815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cd5c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45c8157800 0x7f45c815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55930/0 (socket says 172.21.15.37:55930) 2024-04-26T20:26:32.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cd5c5640 1 -- 172.21.15.37:0/3444211510 learned_addr learned my addr 172.21.15.37:0/3444211510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:32.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cd5c5640 1 -- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45c8073180 msgr2=0x7f45c81357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45c7fff640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c80cefa0 0x7f45c81585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45ccdc4640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45c8073180 0x7f45c81357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cd5c5640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45c8073180 0x7f45c81357a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cd5c5640 1 -- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c80cefa0 msgr2=0x7f45c81585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cd5c5640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c80cefa0 0x7f45c81585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cd5c5640 1 -- 172.21.15.37:0/3444211510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c8158b30 con 0x7f45c8157800 2024-04-26T20:26:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45c7fff640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c80cefa0 0x7f45c81585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.838+0000 7f45cd5c5640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45c8157800 0x7f45c815f830 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c820 tx=0x7f45b800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.839+0000 7f45c57fa640 1 -- 172.21.15.37:0/3444211510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8015070 con 0x7f45c8157800 2024-04-26T20:26:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.839+0000 7f45c57fa640 1 -- 172.21.15.37:0/3444211510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b8007a30 con 0x7f45c8157800 2024-04-26T20:26:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.839+0000 7f45c57fa640 1 -- 172.21.15.37:0/3444211510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8007e20 con 0x7f45c8157800 2024-04-26T20:26:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.839+0000 7f45cf028640 1 -- 172.21.15.37:0/3444211510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c8159930 con 0x7f45c8157800 2024-04-26T20:26:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.839+0000 7f45cf028640 1 -- 172.21.15.37:0/3444211510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c8159ef0 con 0x7f45c8157800 2024-04-26T20:26:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.840+0000 7f45cf028640 1 -- 172.21.15.37:0/3444211510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c806a360 con 0x7f45c8157800 2024-04-26T20:26:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.840+0000 7f45c57fa640 1 -- 172.21.15.37:0/3444211510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f45b8018070 con 0x7f45c8157800 2024-04-26T20:26:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.841+0000 7f45c57fa640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f45b406cbf0 0x7f45b406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.841+0000 7f45c57fa640 1 -- 172.21.15.37:0/3444211510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f45b808bc80 con 0x7f45c8157800 2024-04-26T20:26:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.841+0000 7f45ccdc4640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f45b406cbf0 0x7f45b406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.842+0000 7f45ccdc4640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f45b406cbf0 0x7f45b406f0b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f45c0033000 tx=0x7f45c0009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:32.845+0000 7f45c57fa640 1 -- 172.21.15.37:0/3444211510 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45b8059fe0 con 0x7f45c8157800 2024-04-26T20:26:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.031+0000 7f45cf028640 1 -- 172.21.15.37:0/3444211510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45c8157090 con 0x7f45c8157800 2024-04-26T20:26:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.032+0000 7f45c57fa640 1 -- 172.21.15.37:0/3444211510 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+13032 (secure 0 0 0) 0x7f45b805ee20 con 0x7f45c8157800 2024-04-26T20:26:33.033 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:33.033 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":51,"fsid":"d49b1c18-e1bf-49f2-acda-0904fa7c6684","created":"2024-04-26T20:21:25.185468+0000","modified":"2024-04-26T20:26:29.680659+0000","last_up_change":"2024-04-26T20:21:36.404812+0000","last_in_change":"2024-04-26T20:21:27.357955+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:21:37.350254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b0acb25-b663-44cd-9fec-383b0b04dc80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6809","nonce":2482772026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6811","nonce":2482772026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6815","nonce":2482772026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2482772026},{"type":"v1","addr":"172.21.15.37:6813","nonce":2482772026}]},"public_addr":"172.21.15.37:6809/2482772026","cluster_addr":"172.21.15.37:6811/2482772026","heartbeat_back_addr":"172.21.15.37:6815/2482772026","heartbeat_front_addr":"172.21.15.37:6813/2482772026","state":["exists","up"]},{"osd":1,"uuid":"943f618a-aa4e-4f44-9d8e-f5ddd2eeeda4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6801","nonce":2029719416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6803","nonce":2029719416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6807","nonce":2029719416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2029719416},{"type":"v1","addr":"172.21.15.37:6805","nonce":2029719416}]},"public_addr":"172.21.15.37:6801/2029719416","cluster_addr":"172.21.15.37:6803/2029719416","heartbeat_back_addr":"172.21.15.37:6807/2029719416","heartbeat_front_addr":"172.21.15.37:6805/2029719416","state":["exists","up"]},{"osd":2,"uuid":"ac86bc03-5f61-4c8f-bf48-f0de41624cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6817","nonce":2890277493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6820","nonce":2890277493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6825","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6827","nonce":2890277493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6821","nonce":2890277493},{"type":"v1","addr":"172.21.15.37:6823","nonce":2890277493}]},"public_addr":"172.21.15.37:6817/2890277493","cluster_addr":"172.21.15.37:6820/2890277493","heartbeat_back_addr":"172.21.15.37:6827/2890277493","heartbeat_front_addr":"172.21.15.37:6823/2890277493","state":["exists","up"]},{"osd":3,"uuid":"04591f65-3634-49b3-918d-82dfe0c3c61c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6819","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6822","nonce":2156822987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6826","nonce":2156822987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6831","nonce":2156822987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2156822987},{"type":"v1","addr":"172.21.15.37:6829","nonce":2156822987}]},"public_addr":"172.21.15.37:6822/2156822987","cluster_addr":"172.21.15.37:6826/2156822987","heartbeat_back_addr":"172.21.15.37:6831/2156822987","heartbeat_front_addr":"172.21.15.37:6829/2156822987","state":["exists","up"]},{"osd":4,"uuid":"6099e58a-ea2e-43bf-ac9e-93899bf0093f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6825","nonce":2837585248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6827","nonce":2837585248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6831","nonce":2837585248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2837585248},{"type":"v1","addr":"172.21.15.134:6829","nonce":2837585248}]},"public_addr":"172.21.15.134:6825/2837585248","cluster_addr":"172.21.15.134:6827/2837585248","heartbeat_back_addr":"172.21.15.134:6831/2837585248","heartbeat_front_addr":"172.21.15.134:6829/2837585248","state":["exists","up"]},{"osd":5,"uuid":"43243462-434b-4689-8988-b2a110153fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6809","nonce":1335053617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6811","nonce":1335053617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6815","nonce":1335053617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1335053617},{"type":"v1","addr":"172.21.15.134:6813","nonce":1335053617}]},"public_addr":"172.21.15.134:6809/1335053617","cluster_addr":"172.21.15.134:6811/1335053617","heartbeat_back_addr":"172.21.15.134:6815/1335053617","heartbeat_front_addr":"172.21.15.134:6813/1335053617","state":["exists","up"]},{"osd":6,"uuid":"81d64db1-17fb-4555-b606-592c70879aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6817","nonce":1715563323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6819","nonce":1715563323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6823","nonce":1715563323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1715563323},{"type":"v1","addr":"172.21.15.134:6821","nonce":1715563323}]},"public_addr":"172.21.15.134:6817/1715563323","cluster_addr":"172.21.15.134:6819/1715563323","heartbeat_back_addr":"172.21.15.134:6823/1715563323","heartbeat_front_addr":"172.21.15.134:6821/1715563323","state":["exists","up"]},{"osd":7,"uuid":"dda9aab8-394e-4948-9c5b-f9cd5f9a792c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6801","nonce":4008114040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6803","nonce":4008114040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6807","nonce":4008114040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4008114040},{"type":"v1","addr":"172.21.15.134:6805","nonce":4008114040}]},"public_addr":"172.21.15.134:6801/4008114040","cluster_addr":"172.21.15.134:6803/4008114040","heartbeat_back_addr":"172.21.15.134:6807/4008114040","heartbeat_front_addr":"172.21.15.134:6805/4008114040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.560934+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.468254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.522063+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.497448+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.538002+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.546856+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.451379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:21:32.518510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1915220099":"2024-04-27T20:26:26.618317+0000","172.21.15.37:6834/1915220099":"2024-04-27T20:26:26.618317+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 -- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f45b406cbf0 msgr2=0x7f45b406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f45b406cbf0 0x7f45b406f0b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f45c0033000 tx=0x7f45c0009000 comp rx=0 tx=0).stop 2024-04-26T20:26:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 -- 172.21.15.37:0/3444211510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45c8157800 msgr2=0x7f45c815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45c8157800 0x7f45c815f830 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c820 tx=0x7f45b800cce0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 -- 172.21.15.37:0/3444211510 shutdown_connections 2024-04-26T20:26:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45c8157800 0x7f45c815f830 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c80cefa0 0x7f45c81585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f45b406cbf0 0x7f45b406f0b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 --2- 172.21.15.37:0/3444211510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45c8073180 0x7f45c81357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 -- 172.21.15.37:0/3444211510 >> 172.21.15.37:0/3444211510 conn(0x7f45c80b3b70 msgr2=0x7f45c8071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:33.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 -- 172.21.15.37:0/3444211510 shutdown_connections 2024-04-26T20:26:33.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:33.034+0000 7f45cf028640 1 -- 172.21.15.37:0/3444211510 wait complete. 2024-04-26T20:26:34.047 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T20:26:34.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:26:34.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.152+0000 7fde70112640 1 Processor -- start 2024-04-26T20:26:34.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.152+0000 7fde70112640 1 -- start start 2024-04-26T20:26:34.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde70112640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fde680cc4a0 0x7fde680c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde70112640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fde680cc950 0x7fde680c7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde70112640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde680ced10 con 0x7fde680ce270 2024-04-26T20:26:34.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde70112640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fde680cf920 con 0x7fde680cc4a0 2024-04-26T20:26:34.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde70112640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fde6815d570 con 0x7fde680cc950 2024-04-26T20:26:34.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde6d6ad640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fde680cc4a0 0x7fde680c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde6d6ad640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fde680cc4a0 0x7fde680c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55944/0 (socket says 172.21.15.37:55944) 2024-04-26T20:26:34.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde6d6ad640 1 -- 172.21.15.37:0/715783439 learned_addr learned my addr 172.21.15.37:0/715783439 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:34.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde6e6af640 1 --2- 172.21.15.37:0/715783439 >> v2:172.21.15.134:3300/0 conn(0x7fde680cc950 0x7fde680c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde6d6ad640 1 -- 172.21.15.37:0/715783439 >> v2:172.21.15.134:3300/0 conn(0x7fde680cc950 msgr2=0x7fde680c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde6d6ad640 1 --2- 172.21.15.37:0/715783439 >> v2:172.21.15.134:3300/0 conn(0x7fde680cc950 0x7fde680c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde6d6ad640 1 -- 172.21.15.37:0/715783439 >> v1:172.21.15.134:6789/0 conn(0x7fde680ce270 legacy=0x7fde6815ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde6d6ad640 1 -- 172.21.15.37:0/715783439 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde680ceff0 con 0x7fde680cc4a0 2024-04-26T20:26:34.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.153+0000 7fde6e6af640 1 --2- 172.21.15.37:0/715783439 >> v2:172.21.15.134:3300/0 conn(0x7fde680cc950 0x7fde680c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:34.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.154+0000 7fde6d6ad640 1 --2- 172.21.15.37:0/715783439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde680cc4a0 0x7fde680c6e00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fde680c7b10 tx=0x7fde5802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76962cf520a3c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.154+0000 7fde6ceac640 1 -- 172.21.15.37:0/715783439 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58037070 con 0x7fde680cc4a0 2024-04-26T20:26:34.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.154+0000 7fde6ceac640 1 -- 172.21.15.37:0/715783439 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde58032440 con 0x7fde680cc4a0 2024-04-26T20:26:34.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.154+0000 7fde6ceac640 1 -- 172.21.15.37:0/715783439 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58032810 con 0x7fde680cc4a0 2024-04-26T20:26:34.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.155+0000 7fde70112640 1 -- 172.21.15.37:0/715783439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde680cc4a0 msgr2=0x7fde680c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.155+0000 7fde70112640 1 --2- 172.21.15.37:0/715783439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde680cc4a0 0x7fde680c6e00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fde680c7b10 tx=0x7fde5802a230 comp rx=0 tx=0).stop 2024-04-26T20:26:34.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.155+0000 7fde70112640 1 -- 172.21.15.37:0/715783439 shutdown_connections 2024-04-26T20:26:34.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.155+0000 7fde70112640 1 --2- 172.21.15.37:0/715783439 >> v2:172.21.15.134:3300/0 conn(0x7fde680cc950 0x7fde680c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.155+0000 7fde70112640 1 --2- 172.21.15.37:0/715783439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde680cc4a0 0x7fde680c6e00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.155+0000 7fde70112640 1 -- 172.21.15.37:0/715783439 >> 172.21.15.37:0/715783439 conn(0x7fde680b2c60 msgr2=0x7fde680b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.155+0000 7fde70112640 1 -- 172.21.15.37:0/715783439 shutdown_connections 2024-04-26T20:26:34.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.155+0000 7fde70112640 1 -- 172.21.15.37:0/715783439 wait complete. 2024-04-26T20:26:34.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.156+0000 7fde70112640 1 Processor -- start 2024-04-26T20:26:34.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.156+0000 7fde70112640 1 -- start start 2024-04-26T20:26:34.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde70112640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde680cc4a0 0x7fde680b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde70112640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde680cc950 0x7fde681e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde70112640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde680ce270 0x7fde681e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde70112640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde680ce650 con 0x7fde680cc4a0 2024-04-26T20:26:34.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde70112640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde680cf5b0 con 0x7fde680cc950 2024-04-26T20:26:34.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde70112640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fde680ce9e0 con 0x7fde680ce270 2024-04-26T20:26:34.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde6deae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde680cc4a0 0x7fde680b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde6deae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde680cc4a0 0x7fde680b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55952/0 (socket says 172.21.15.37:55952) 2024-04-26T20:26:34.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde6deae640 1 -- 172.21.15.37:0/1007147073 learned_addr learned my addr 172.21.15.37:0/1007147073 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:34.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde6deae640 1 -- 172.21.15.37:0/1007147073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde680ce270 msgr2=0x7fde681e70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde6deae640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde680ce270 0x7fde681e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde6deae640 1 -- 172.21.15.37:0/1007147073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde680cc950 msgr2=0x7fde681e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde6deae640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde680cc950 0x7fde681e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde6deae640 1 -- 172.21.15.37:0/1007147073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde681e8890 con 0x7fde680cc4a0 2024-04-26T20:26:34.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.157+0000 7fde6deae640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde680cc4a0 0x7fde680b20c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fde6400b420 tx=0x7fde6400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.158+0000 7fde5e7fc640 1 -- 172.21.15.37:0/1007147073 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64011830 con 0x7fde680cc4a0 2024-04-26T20:26:34.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.158+0000 7fde5e7fc640 1 -- 172.21.15.37:0/1007147073 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde64016070 con 0x7fde680cc4a0 2024-04-26T20:26:34.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.158+0000 7fde70112640 1 -- 172.21.15.37:0/1007147073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde681e8b30 con 0x7fde680cc4a0 2024-04-26T20:26:34.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.158+0000 7fde5e7fc640 1 -- 172.21.15.37:0/1007147073 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64011c10 con 0x7fde680cc4a0 2024-04-26T20:26:34.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.158+0000 7fde70112640 1 -- 172.21.15.37:0/1007147073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde681e90f0 con 0x7fde680cc4a0 2024-04-26T20:26:34.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.158+0000 7fde70112640 1 -- 172.21.15.37:0/1007147073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fde681e7620 con 0x7fde680cc4a0 2024-04-26T20:26:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.159+0000 7fde5e7fc640 1 -- 172.21.15.37:0/1007147073 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fde6401b460 con 0x7fde680cc4a0 2024-04-26T20:26:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.160+0000 7fde5e7fc640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fde4c06cbf0 0x7fde4c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.160+0000 7fde5e7fc640 1 -- 172.21.15.37:0/1007147073 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fde6408c4e0 con 0x7fde680cc4a0 2024-04-26T20:26:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.160+0000 7fde5e7fc640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] conn(0x7fde4c072ab0 0x7fde4c074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.160+0000 7fde5e7fc640 1 -- 172.21.15.37:0/1007147073 --> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde6400fe50 con 0x7fde4c072ab0 2024-04-26T20:26:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.160+0000 7fde5e7fc640 1 -- 172.21.15.37:0/1007147073 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fde6408c910 con 0x7fde680cc4a0 2024-04-26T20:26:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.160+0000 7fde6d6ad640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fde4c06cbf0 0x7fde4c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.160+0000 7fde6e6af640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] conn(0x7fde4c072ab0 0x7fde4c074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.160+0000 7fde6e6af640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] conn(0x7fde4c072ab0 0x7fde4c074ed0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.160+0000 7fde6d6ad640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fde4c06cbf0 0x7fde4c06f0b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fde58034000 tx=0x7fde5802c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.161+0000 7fde5e7fc640 1 -- 172.21.15.37:0/1007147073 <== osd.0 v2:172.21.15.37:6808/2482772026 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fde6400fe50 con 0x7fde4c072ab0 2024-04-26T20:26:34.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.174+0000 7fde70112640 1 -- 172.21.15.37:0/1007147073 --> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fde6806a360 con 0x7fde4c072ab0 2024-04-26T20:26:34.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.174+0000 7fde5e7fc640 1 -- 172.21.15.37:0/1007147073 <== osd.0 v2:172.21.15.37:6808/2482772026 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fde6806a360 con 0x7fde4c072ab0 2024-04-26T20:26:34.176 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:26:34.176 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:26:34.176 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:26:34.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.175+0000 7fde70112640 1 -- 172.21.15.37:0/1007147073 >> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] conn(0x7fde4c072ab0 msgr2=0x7fde4c074ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.175+0000 7fde70112640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.37:6808/2482772026,v1:172.21.15.37:6809/2482772026] conn(0x7fde4c072ab0 0x7fde4c074ed0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.175+0000 7fde70112640 1 -- 172.21.15.37:0/1007147073 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fde4c06cbf0 msgr2=0x7fde4c06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.175+0000 7fde70112640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fde4c06cbf0 0x7fde4c06f0b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fde58034000 tx=0x7fde5802c000 comp rx=0 tx=0).stop 2024-04-26T20:26:34.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.175+0000 7fde70112640 1 -- 172.21.15.37:0/1007147073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde680cc4a0 msgr2=0x7fde680b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.175+0000 7fde70112640 1 --2- 172.21.15.37:0/1007147073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde680cc4a0 0x7fde680b20c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fde6400b420 tx=0x7fde6400b8e0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.175+0000 7fde6e6af640 1 -- 172.21.15.37:0/1007147073 reap_dead start 2024-04-26T20:26:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.175+0000 7fde70112640 1 -- 172.21.15.37:0/1007147073 shutdown_connections 2024-04-26T20:26:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.175+0000 7fde70112640 1 -- 172.21.15.37:0/1007147073 >> 172.21.15.37:0/1007147073 conn(0x7fde680b2c60 msgr2=0x7fde680d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.175+0000 7fde70112640 1 -- 172.21.15.37:0/1007147073 shutdown_connections 2024-04-26T20:26:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.175+0000 7fde70112640 1 -- 172.21.15.37:0/1007147073 wait complete. 2024-04-26T20:26:34.186 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T20:26:34.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.288+0000 7f66dc991640 1 Processor -- start 2024-04-26T20:26:34.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.288+0000 7f66dc991640 1 -- start start 2024-04-26T20:26:34.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66dc991640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f66d809ba50 0x7f66d80b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66dc991640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d80d23e0 0x7f66d815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66dc991640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66d80cd650 con 0x7f66d806d560 2024-04-26T20:26:34.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66dc991640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f66d80cd8f0 con 0x7f66d809ba50 2024-04-26T20:26:34.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66dc991640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f66d80cdbd0 con 0x7f66d80d23e0 2024-04-26T20:26:34.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66cffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f66d806d560 0x7f66d809be30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48114/0 (socket says 172.21.15.37:48114) 2024-04-26T20:26:34.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66cffff640 1 -- 172.21.15.37:0/1148072663 learned_addr learned my addr 172.21.15.37:0/1148072663 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:34.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66d6d9d640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d80d23e0 0x7f66d815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66d659c640 1 --2- 172.21.15.37:0/1148072663 >> v2:172.21.15.134:3300/0 conn(0x7f66d809ba50 0x7f66d80b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2049239586 0 0) 0x7f66d80cd650 con 0x7f66d806d560 2024-04-26T20:26:34.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66c4003520 con 0x7f66d806d560 2024-04-26T20:26:34.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1609300976 0 0) 0x7f66c4003520 con 0x7f66d806d560 2024-04-26T20:26:34.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d80d23e0 msgr2=0x7f66d815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66d5d9b640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d80d23e0 0x7f66d815ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.289+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 >> v2:172.21.15.134:3300/0 conn(0x7f66d809ba50 msgr2=0x7f66d80b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d5d9b640 1 --2- 172.21.15.37:0/1148072663 >> v2:172.21.15.134:3300/0 conn(0x7f66d809ba50 0x7f66d80b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d80cdeb0 con 0x7f66d806d560 2024-04-26T20:26:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f66c0001670 con 0x7f66d806d560 2024-04-26T20:26:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d659c640 1 --2- 172.21.15.37:0/1148072663 >> v2:172.21.15.134:3300/0 conn(0x7f66d809ba50 0x7f66d80b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 >> v1:172.21.15.37:6789/0 conn(0x7f66d806d560 legacy=0x7f66d809be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d5d9b640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66c40039b0 0x7f66c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d5d9b640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66c4004800 0x7f66c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d5d9b640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d80d23e0 0x7f66c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f66d80cd8f0 con 0x7f66c40039b0 2024-04-26T20:26:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66d80cdbd0 con 0x7f66c4004800 2024-04-26T20:26:34.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f66c400b9c0 con 0x7f66d80d23e0 2024-04-26T20:26:34.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d659c640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66c40039b0 0x7f66c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d6d9d640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d80d23e0 0x7f66c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66cffff640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66c4004800 0x7f66c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d659c640 1 -- 172.21.15.37:0/1148072663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d80d23e0 msgr2=0x7f66c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d659c640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d80d23e0 0x7f66c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d659c640 1 -- 172.21.15.37:0/1148072663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66c4004800 msgr2=0x7f66c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d659c640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66c4004800 0x7f66c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.290+0000 7f66d659c640 1 -- 172.21.15.37:0/1148072663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c4003520 con 0x7f66c40039b0 2024-04-26T20:26:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.291+0000 7f66d659c640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66c40039b0 0x7f66c4003dd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009c90 tx=0x7f66c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78c297f27c2b4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.291+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8036070 con 0x7f66c40039b0 2024-04-26T20:26:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.291+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c8031400 con 0x7f66c40039b0 2024-04-26T20:26:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.291+0000 7f66d5d9b640 1 -- 172.21.15.37:0/1148072663 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8009860 con 0x7f66c40039b0 2024-04-26T20:26:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.292+0000 7f66dc991640 1 -- 172.21.15.37:0/1148072663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66c40039b0 msgr2=0x7f66c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.292+0000 7f66dc991640 1 --2- 172.21.15.37:0/1148072663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66c40039b0 0x7f66c4003dd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009c90 tx=0x7f66c4005830 comp rx=0 tx=0).stop 2024-04-26T20:26:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.292+0000 7f66d6d9d640 1 -- 172.21.15.37:0/1148072663 reap_dead start 2024-04-26T20:26:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.292+0000 7f66dc991640 1 -- 172.21.15.37:0/1148072663 shutdown_connections 2024-04-26T20:26:34.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.292+0000 7f66dc991640 1 -- 172.21.15.37:0/1148072663 >> 172.21.15.37:0/1148072663 conn(0x7f66d80b4a50 msgr2=0x7f66d80b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.292+0000 7f66dc991640 1 -- 172.21.15.37:0/1148072663 shutdown_connections 2024-04-26T20:26:34.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.292+0000 7f66dc991640 1 -- 172.21.15.37:0/1148072663 wait complete. 2024-04-26T20:26:34.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.293+0000 7f66dc991640 1 Processor -- start 2024-04-26T20:26:34.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.293+0000 7f66dc991640 1 -- start start 2024-04-26T20:26:34.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66dc991640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66c40039b0 0x7f66d81e1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66dc991640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d81e1800 0x7f66d80b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66dc991640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d81e1c80 0x7f66d80b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66dc991640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f66d80d2860 con 0x7f66d81e1c80 2024-04-26T20:26:34.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66dc991640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66d80b4790 con 0x7f66d81e1800 2024-04-26T20:26:34.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66dc991640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f66d80d29e0 con 0x7f66c40039b0 2024-04-26T20:26:34.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d81e1c80 0x7f66d80b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d81e1c80 0x7f66d80b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55976/0 (socket says 172.21.15.37:55976) 2024-04-26T20:26:34.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66d6d9d640 1 -- 172.21.15.37:0/2400384771 learned_addr learned my addr 172.21.15.37:0/2400384771 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:34.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66d659c640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66c40039b0 0x7f66d81e1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66cffff640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d81e1800 0x7f66d80b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.294+0000 7f66d6d9d640 1 -- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66c40039b0 msgr2=0x7f66d81e1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.295+0000 7f66d6d9d640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66c40039b0 0x7f66d81e1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.295+0000 7f66d6d9d640 1 -- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d81e1800 msgr2=0x7f66d80b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.295+0000 7f66d6d9d640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d81e1800 0x7f66d80b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.295+0000 7f66d6d9d640 1 -- 172.21.15.37:0/2400384771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d81e2a80 con 0x7f66d81e1c80 2024-04-26T20:26:34.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.295+0000 7f66d6d9d640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d81e1c80 0x7f66d80b13f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f66d80b9010 tx=0x7f66d000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.295+0000 7f66ceffd640 1 -- 172.21.15.37:0/2400384771 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0011e60 con 0x7f66d81e1c80 2024-04-26T20:26:34.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.295+0000 7f66ceffd640 1 -- 172.21.15.37:0/2400384771 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d0015430 con 0x7f66d81e1c80 2024-04-26T20:26:34.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.295+0000 7f66dc991640 1 -- 172.21.15.37:0/2400384771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d81e15b0 con 0x7f66d81e1c80 2024-04-26T20:26:34.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.295+0000 7f66ceffd640 1 -- 172.21.15.37:0/2400384771 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0015800 con 0x7f66d81e1c80 2024-04-26T20:26:34.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.295+0000 7f66dc991640 1 -- 172.21.15.37:0/2400384771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d81e24c0 con 0x7f66d81e1c80 2024-04-26T20:26:34.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.296+0000 7f66dc991640 1 -- 172.21.15.37:0/2400384771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d80693e0 con 0x7f66d81e1c80 2024-04-26T20:26:34.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.297+0000 7f66ceffd640 1 -- 172.21.15.37:0/2400384771 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f66d00159a0 con 0x7f66d81e1c80 2024-04-26T20:26:34.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.297+0000 7f66ceffd640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f66b806cbc0 0x7f66b806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.297+0000 7f66ceffd640 1 -- 172.21.15.37:0/2400384771 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f66d008cf10 con 0x7f66d81e1c80 2024-04-26T20:26:34.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.297+0000 7f66d659c640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f66b806cbc0 0x7f66b806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.298+0000 7f66d659c640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f66b806cbc0 0x7f66b806f080 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f66c8033000 tx=0x7f66c8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.302+0000 7f66ceffd640 1 -- 172.21.15.37:0/2400384771 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66d0019030 con 0x7f66d81e1c80 2024-04-26T20:26:34.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.488+0000 7f66dc991640 1 -- 172.21.15.37:0/2400384771 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f66d80cf290 con 0x7f66b806cbc0 2024-04-26T20:26:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.489+0000 7f66ceffd640 1 -- 172.21.15.37:0/2400384771 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f66d80cf290 con 0x7f66b806cbc0 2024-04-26T20:26:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T20:26:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 -- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f66b806cbc0 msgr2=0x7f66b806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f66b806cbc0 0x7f66b806f080 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f66c8033000 tx=0x7f66c8008030 comp rx=0 tx=0).stop 2024-04-26T20:26:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 -- 172.21.15.37:0/2400384771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d81e1c80 msgr2=0x7f66d80b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d81e1c80 0x7f66d80b13f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f66d80b9010 tx=0x7f66d000f580 comp rx=0 tx=0).stop 2024-04-26T20:26:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 -- 172.21.15.37:0/2400384771 shutdown_connections 2024-04-26T20:26:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d81e1c80 0x7f66d80b13f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d81e1800 0x7f66d80b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f66b806cbc0 0x7f66b806f080 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 --2- 172.21.15.37:0/2400384771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66c40039b0 0x7f66d81e1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 -- 172.21.15.37:0/2400384771 >> 172.21.15.37:0/2400384771 conn(0x7f66d80b4a50 msgr2=0x7f66d80709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 -- 172.21.15.37:0/2400384771 shutdown_connections 2024-04-26T20:26:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.491+0000 7f66dc991640 1 -- 172.21.15.37:0/2400384771 wait complete. 2024-04-26T20:26:34.503 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T20:26:34.503 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:26:34.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.607+0000 7f77fb951640 1 Processor -- start 2024-04-26T20:26:34.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.607+0000 7f77fb951640 1 -- start start 2024-04-26T20:26:34.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77fb951640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f77f40ce270 0x7f77f415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77fb951640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f77f40c7510 0x7f77f40cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77fb951640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77f40ced10 con 0x7f77f40cc4a0 2024-04-26T20:26:34.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77fb951640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f77f40cf920 con 0x7f77f40c7510 2024-04-26T20:26:34.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77fb951640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f77f40ceff0 con 0x7f77f40ce270 2024-04-26T20:26:34.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77f9eee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f77f40c7510 0x7f77f40cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77f9eee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f77f40c7510 0x7f77f40cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55984/0 (socket says 172.21.15.37:55984) 2024-04-26T20:26:34.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77f9eee640 1 -- 172.21.15.37:0/3415896745 learned_addr learned my addr 172.21.15.37:0/3415896745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:34.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77f96ed640 1 --2- 172.21.15.37:0/3415896745 >> v2:172.21.15.134:3300/0 conn(0x7f77f40ce270 0x7f77f415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77f9eee640 1 -- 172.21.15.37:0/3415896745 >> v2:172.21.15.134:3300/0 conn(0x7f77f40ce270 msgr2=0x7f77f415ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77f9eee640 1 --2- 172.21.15.37:0/3415896745 >> v2:172.21.15.134:3300/0 conn(0x7f77f40ce270 0x7f77f415ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77f9eee640 1 -- 172.21.15.37:0/3415896745 >> v1:172.21.15.134:6789/0 conn(0x7f77f40cc4a0 legacy=0x7f77f40c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.608+0000 7f77f9eee640 1 -- 172.21.15.37:0/3415896745 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f40cf2d0 con 0x7f77f40c7510 2024-04-26T20:26:34.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.609+0000 7f77f9eee640 1 --2- 172.21.15.37:0/3415896745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77f40c7510 0x7f77f40cfdb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f77f40c7e90 tx=0x7f77e402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd46440e4b39332 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.609+0000 7f77f3fff640 1 -- 172.21.15.37:0/3415896745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4036070 con 0x7f77f40c7510 2024-04-26T20:26:34.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.609+0000 7f77f3fff640 1 -- 172.21.15.37:0/3415896745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e4031600 con 0x7f77f40c7510 2024-04-26T20:26:34.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.609+0000 7f77f3fff640 1 -- 172.21.15.37:0/3415896745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e40319d0 con 0x7f77f40c7510 2024-04-26T20:26:34.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.610+0000 7f77fb951640 1 -- 172.21.15.37:0/3415896745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77f40c7510 msgr2=0x7f77f40cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.610+0000 7f77fb951640 1 --2- 172.21.15.37:0/3415896745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77f40c7510 0x7f77f40cfdb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f77f40c7e90 tx=0x7f77e402a6f0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.610+0000 7f77fb951640 1 -- 172.21.15.37:0/3415896745 shutdown_connections 2024-04-26T20:26:34.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.610+0000 7f77fb951640 1 --2- 172.21.15.37:0/3415896745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77f40c7510 0x7f77f40cfdb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.610+0000 7f77fb951640 1 --2- 172.21.15.37:0/3415896745 >> v2:172.21.15.134:3300/0 conn(0x7f77f40ce270 0x7f77f415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.610+0000 7f77fb951640 1 -- 172.21.15.37:0/3415896745 >> 172.21.15.37:0/3415896745 conn(0x7f77f40b2c60 msgr2=0x7f77f40b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.610+0000 7f77fb951640 1 -- 172.21.15.37:0/3415896745 shutdown_connections 2024-04-26T20:26:34.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.610+0000 7f77fb951640 1 -- 172.21.15.37:0/3415896745 wait complete. 2024-04-26T20:26:34.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.611+0000 7f77fb951640 1 Processor -- start 2024-04-26T20:26:34.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.611+0000 7f77fb951640 1 -- start start 2024-04-26T20:26:34.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77fb951640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77f40c7510 0x7f77f414eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77fb951640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77f40cc4a0 0x7f77f40738a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77fb951640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77f40ce270 0x7f77f40c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77fb951640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77f40ce9e0 con 0x7f77f40cc4a0 2024-04-26T20:26:34.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77fb951640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77f40ce650 con 0x7f77f40c7510 2024-04-26T20:26:34.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77fb951640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f77f406f3a0 con 0x7f77f40ce270 2024-04-26T20:26:34.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77f8eec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77f40cc4a0 0x7f77f40738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77f8eec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77f40cc4a0 0x7f77f40738a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55992/0 (socket says 172.21.15.37:55992) 2024-04-26T20:26:34.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77f8eec640 1 -- 172.21.15.37:0/1888382452 learned_addr learned my addr 172.21.15.37:0/1888382452 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:34.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77f96ed640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77f40c7510 0x7f77f414eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77f9eee640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77f40ce270 0x7f77f40c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77f8eec640 1 -- 172.21.15.37:0/1888382452 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77f40ce270 msgr2=0x7f77f40c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77f8eec640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77f40ce270 0x7f77f40c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77f8eec640 1 -- 172.21.15.37:0/1888382452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77f40c7510 msgr2=0x7f77f414eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77f8eec640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77f40c7510 0x7f77f414eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77f8eec640 1 -- 172.21.15.37:0/1888382452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f41f5a40 con 0x7f77f40cc4a0 2024-04-26T20:26:34.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.612+0000 7f77f8eec640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77f40cc4a0 0x7f77f40738a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f77e800c460 tx=0x7f77e800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.613+0000 7f77f1ffb640 1 -- 172.21.15.37:0/1888382452 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8016020 con 0x7f77f40cc4a0 2024-04-26T20:26:34.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.613+0000 7f77f1ffb640 1 -- 172.21.15.37:0/1888382452 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e8012870 con 0x7f77f40cc4a0 2024-04-26T20:26:34.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.613+0000 7f77fb951640 1 -- 172.21.15.37:0/1888382452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f41f5d20 con 0x7f77f40cc4a0 2024-04-26T20:26:34.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.613+0000 7f77f1ffb640 1 -- 172.21.15.37:0/1888382452 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8012c60 con 0x7f77f40cc4a0 2024-04-26T20:26:34.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.613+0000 7f77fb951640 1 -- 172.21.15.37:0/1888382452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f41f62e0 con 0x7f77f40cc4a0 2024-04-26T20:26:34.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.614+0000 7f77fb951640 1 -- 172.21.15.37:0/1888382452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77f41f66c0 con 0x7f77f40cc4a0 2024-04-26T20:26:34.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.614+0000 7f77f1ffb640 1 -- 172.21.15.37:0/1888382452 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f77e8019020 con 0x7f77f40cc4a0 2024-04-26T20:26:34.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.615+0000 7f77f1ffb640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f77e006cbc0 0x7f77e006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.615+0000 7f77f1ffb640 1 -- 172.21.15.37:0/1888382452 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f77e808bcf0 con 0x7f77f40cc4a0 2024-04-26T20:26:34.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.615+0000 7f77f1ffb640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] conn(0x7f77e0072a80 0x7f77e0074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.615+0000 7f77f1ffb640 1 -- 172.21.15.37:0/1888382452 --> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77e0075550 con 0x7f77e0072a80 2024-04-26T20:26:34.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.615+0000 7f77f1ffb640 1 -- 172.21.15.37:0/1888382452 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f77e8091380 con 0x7f77f40cc4a0 2024-04-26T20:26:34.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.615+0000 7f77f96ed640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f77e006cbc0 0x7f77e006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.615+0000 7f77f9eee640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] conn(0x7f77e0072a80 0x7f77e0074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.615+0000 7f77f9eee640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] conn(0x7f77e0072a80 0x7f77e0074ea0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.615+0000 7f77f96ed640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f77e006cbc0 0x7f77e006f080 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f77ec006fd0 tx=0x7f77ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.616+0000 7f77f1ffb640 1 -- 172.21.15.37:0/1888382452 <== osd.1 v2:172.21.15.37:6800/2029719416 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77e0075550 con 0x7f77e0072a80 2024-04-26T20:26:34.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.629+0000 7f77fb951640 1 -- 172.21.15.37:0/1888382452 --> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f77f40b2400 con 0x7f77e0072a80 2024-04-26T20:26:34.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77f1ffb640 1 -- 172.21.15.37:0/1888382452 <== osd.1 v2:172.21.15.37:6800/2029719416 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f77f40b2400 con 0x7f77e0072a80 2024-04-26T20:26:34.631 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:26:34.631 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:26:34.631 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:26:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77fb951640 1 -- 172.21.15.37:0/1888382452 >> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] conn(0x7f77e0072a80 msgr2=0x7f77e0074ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77fb951640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.37:6800/2029719416,v1:172.21.15.37:6801/2029719416] conn(0x7f77e0072a80 0x7f77e0074ea0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77fb951640 1 -- 172.21.15.37:0/1888382452 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f77e006cbc0 msgr2=0x7f77e006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77fb951640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f77e006cbc0 0x7f77e006f080 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f77ec006fd0 tx=0x7f77ec008040 comp rx=0 tx=0).stop 2024-04-26T20:26:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77fb951640 1 -- 172.21.15.37:0/1888382452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77f40cc4a0 msgr2=0x7f77f40738a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77fb951640 1 --2- 172.21.15.37:0/1888382452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77f40cc4a0 0x7f77f40738a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f77e800c460 tx=0x7f77e800c920 comp rx=0 tx=0).stop 2024-04-26T20:26:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77f9eee640 1 -- 172.21.15.37:0/1888382452 reap_dead start 2024-04-26T20:26:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77fb951640 1 -- 172.21.15.37:0/1888382452 shutdown_connections 2024-04-26T20:26:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77fb951640 1 -- 172.21.15.37:0/1888382452 >> 172.21.15.37:0/1888382452 conn(0x7f77f40b2c60 msgr2=0x7f77f40c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77fb951640 1 -- 172.21.15.37:0/1888382452 shutdown_connections 2024-04-26T20:26:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.630+0000 7f77fb951640 1 -- 172.21.15.37:0/1888382452 wait complete. 2024-04-26T20:26:34.641 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T20:26:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.741+0000 7fa8cb740640 1 Processor -- start 2024-04-26T20:26:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.741+0000 7fa8cb740640 1 -- start start 2024-04-26T20:26:34.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.742+0000 7fa8cb740640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa8c4162b40 0x7fa8c40d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.742+0000 7fa8cb740640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa8c40d3c80 0x7fa8c40d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.742+0000 7fa8cb740640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c40d0210 con 0x7fa8c40d3820 2024-04-26T20:26:34.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.742+0000 7fa8cb740640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa8c40d04f0 con 0x7fa8c4162b40 2024-04-26T20:26:34.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.742+0000 7fa8cb740640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa8c40d07d0 con 0x7fa8c40d3c80 2024-04-26T20:26:34.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.742+0000 7fa8c94dc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa8c4162b40 0x7fa8c40d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.742+0000 7fa8c94dc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa8c4162b40 0x7fa8c40d5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56008/0 (socket says 172.21.15.37:56008) 2024-04-26T20:26:34.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.742+0000 7fa8c8cdb640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa8c40d3820 0x7fa8c40d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48124/0 (socket says 172.21.15.37:48124) 2024-04-26T20:26:34.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.742+0000 7fa8c94dc640 1 -- 172.21.15.37:0/3691140940 learned_addr learned my addr 172.21.15.37:0/3691140940 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:34.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.743+0000 7fa8c9cdd640 1 --2- 172.21.15.37:0/3691140940 >> v2:172.21.15.134:3300/0 conn(0x7fa8c40d3c80 0x7fa8c40d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.743+0000 7fa8c94dc640 1 -- 172.21.15.37:0/3691140940 >> v2:172.21.15.134:3300/0 conn(0x7fa8c40d3c80 msgr2=0x7fa8c40d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.743+0000 7fa8c94dc640 1 --2- 172.21.15.37:0/3691140940 >> v2:172.21.15.134:3300/0 conn(0x7fa8c40d3c80 0x7fa8c40d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.743+0000 7fa8c94dc640 1 -- 172.21.15.37:0/3691140940 >> v1:172.21.15.37:6789/0 conn(0x7fa8c40d3820 legacy=0x7fa8c40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.743+0000 7fa8c94dc640 1 -- 172.21.15.37:0/3691140940 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c40d0ab0 con 0x7fa8c4162b40 2024-04-26T20:26:34.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.743+0000 7fa8bbfff640 1 -- 172.21.15.37:0/3691140940 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2489361954 0 0) 0x7fa8c40d0210 con 0x7fa8c40d3820 2024-04-26T20:26:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.743+0000 7fa8c9cdd640 1 --2- 172.21.15.37:0/3691140940 >> v2:172.21.15.134:3300/0 conn(0x7fa8c40d3c80 0x7fa8c40d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:34.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.743+0000 7fa8c94dc640 1 --2- 172.21.15.37:0/3691140940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4162b40 0x7fa8c40d5bd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa8c409ec60 tx=0x7fa8bc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619c8198cd902bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.744+0000 7fa8bbfff640 1 -- 172.21.15.37:0/3691140940 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc037070 con 0x7fa8c4162b40 2024-04-26T20:26:34.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.744+0000 7fa8bbfff640 1 -- 172.21.15.37:0/3691140940 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8bc02ae50 con 0x7fa8c4162b40 2024-04-26T20:26:34.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.744+0000 7fa8bbfff640 1 -- 172.21.15.37:0/3691140940 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc032860 con 0x7fa8c4162b40 2024-04-26T20:26:34.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.744+0000 7fa8cb740640 1 -- 172.21.15.37:0/3691140940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4162b40 msgr2=0x7fa8c40d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.744+0000 7fa8cb740640 1 --2- 172.21.15.37:0/3691140940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4162b40 0x7fa8c40d5bd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa8c409ec60 tx=0x7fa8bc02a300 comp rx=0 tx=0).stop 2024-04-26T20:26:34.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.744+0000 7fa8cb740640 1 -- 172.21.15.37:0/3691140940 shutdown_connections 2024-04-26T20:26:34.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.744+0000 7fa8cb740640 1 --2- 172.21.15.37:0/3691140940 >> v2:172.21.15.134:3300/0 conn(0x7fa8c40d3c80 0x7fa8c40d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.744+0000 7fa8cb740640 1 --2- 172.21.15.37:0/3691140940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4162b40 0x7fa8c40d5bd0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.744+0000 7fa8cb740640 1 -- 172.21.15.37:0/3691140940 >> 172.21.15.37:0/3691140940 conn(0x7fa8c409e550 msgr2=0x7fa8c40d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.744+0000 7fa8cb740640 1 -- 172.21.15.37:0/3691140940 shutdown_connections 2024-04-26T20:26:34.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.744+0000 7fa8cb740640 1 -- 172.21.15.37:0/3691140940 wait complete. 2024-04-26T20:26:34.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.745+0000 7fa8cb740640 1 Processor -- start 2024-04-26T20:26:34.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.746+0000 7fa8cb740640 1 -- start start 2024-04-26T20:26:34.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.746+0000 7fa8cb740640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8c40d3820 0x7fa8c40b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.746+0000 7fa8cb740640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c40d3c80 0x7fa8c40c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.746+0000 7fa8cb740640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8c4162b40 0x7fa8c40c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.746+0000 7fa8cb740640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c40b5bb0 con 0x7fa8c40d3c80 2024-04-26T20:26:34.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.746+0000 7fa8cb740640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c40d4110 con 0x7fa8c4162b40 2024-04-26T20:26:34.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.746+0000 7fa8cb740640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c40d6110 con 0x7fa8c40d3820 2024-04-26T20:26:34.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.746+0000 7fa8c8cdb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c40d3c80 0x7fa8c40c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.746+0000 7fa8c8cdb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c40d3c80 0x7fa8c40c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56010/0 (socket says 172.21.15.37:56010) 2024-04-26T20:26:34.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.746+0000 7fa8c8cdb640 1 -- 172.21.15.37:0/1936811165 learned_addr learned my addr 172.21.15.37:0/1936811165 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:34.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8c94dc640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8c40d3820 0x7fa8c40b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8c8cdb640 1 -- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8c40d3820 msgr2=0x7fa8c40b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8c8cdb640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8c40d3820 0x7fa8c40b7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8c9cdd640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8c4162b40 0x7fa8c40c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8c8cdb640 1 -- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8c4162b40 msgr2=0x7fa8c40c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8c8cdb640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8c4162b40 0x7fa8c40c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8c8cdb640 1 -- 172.21.15.37:0/1936811165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c4156070 con 0x7fa8c40d3c80 2024-04-26T20:26:34.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8c9cdd640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8c4162b40 0x7fa8c40c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:34.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8c8cdb640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c40d3c80 0x7fa8c40c8b50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400c430 tx=0x7fa8b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8b9ffb640 1 -- 172.21.15.37:0/1936811165 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40074e0 con 0x7fa8c40d3c80 2024-04-26T20:26:34.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8b9ffb640 1 -- 172.21.15.37:0/1936811165 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b40047e0 con 0x7fa8c40d3c80 2024-04-26T20:26:34.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8cb740640 1 -- 172.21.15.37:0/1936811165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c40bb0f0 con 0x7fa8c40d3c80 2024-04-26T20:26:34.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8b9ffb640 1 -- 172.21.15.37:0/1936811165 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4004bd0 con 0x7fa8c40d3c80 2024-04-26T20:26:34.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.747+0000 7fa8cb740640 1 -- 172.21.15.37:0/1936811165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c40ce0f0 con 0x7fa8c40d3c80 2024-04-26T20:26:34.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.748+0000 7fa8cb740640 1 -- 172.21.15.37:0/1936811165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c406a360 con 0x7fa8c40d3c80 2024-04-26T20:26:34.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.748+0000 7fa8b9ffb640 1 -- 172.21.15.37:0/1936811165 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fa8b4011070 con 0x7fa8c40d3c80 2024-04-26T20:26:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.749+0000 7fa8b9ffb640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa8a806cbc0 0x7fa8a806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.749+0000 7fa8b9ffb640 1 -- 172.21.15.37:0/1936811165 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fa8b408ba80 con 0x7fa8c40d3c80 2024-04-26T20:26:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.749+0000 7fa8c94dc640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa8a806cbc0 0x7fa8a806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.750+0000 7fa8c94dc640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa8a806cbc0 0x7fa8a806f080 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc034000 tx=0x7fa8bc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.753+0000 7fa8b9ffb640 1 -- 172.21.15.37:0/1936811165 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa8b408de20 con 0x7fa8c40d3c80 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.938+0000 7fa8cb740640 1 -- 172.21.15.37:0/1936811165 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fa8c406dfb0 con 0x7fa8a806cbc0 2024-04-26T20:26:34.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.939+0000 7fa8b9ffb640 1 -- 172.21.15.37:0/1936811165 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa8c406dfb0 con 0x7fa8a806cbc0 2024-04-26T20:26:34.940 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 -- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa8a806cbc0 msgr2=0x7fa8a806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa8a806cbc0 0x7fa8a806f080 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc034000 tx=0x7fa8bc02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 -- 172.21.15.37:0/1936811165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c40d3c80 msgr2=0x7fa8c40c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c40d3c80 0x7fa8c40c8b50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400c430 tx=0x7fa8b400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 -- 172.21.15.37:0/1936811165 shutdown_connections 2024-04-26T20:26:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8c4162b40 0x7fa8c40c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c40d3c80 0x7fa8c40c8b50 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa8a806cbc0 0x7fa8a806f080 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 --2- 172.21.15.37:0/1936811165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8c40d3820 0x7fa8c40b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 -- 172.21.15.37:0/1936811165 >> 172.21.15.37:0/1936811165 conn(0x7fa8c409e550 msgr2=0x7fa8c40b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 -- 172.21.15.37:0/1936811165 shutdown_connections 2024-04-26T20:26:34.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:34.941+0000 7fa8cb740640 1 -- 172.21.15.37:0/1936811165 wait complete. 2024-04-26T20:26:34.952 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T20:26:34.953 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:26:35.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.056+0000 7f264a7e4640 1 Processor -- start 2024-04-26T20:26:35.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.056+0000 7f264a7e4640 1 -- start start 2024-04-26T20:26:35.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.057+0000 7f264a7e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440d1950 0x7f26440cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.057+0000 7f264a7e4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2644160390 0x7f26440d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.057+0000 7f264a7e4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264415d320 con 0x7f2644162ee0 2024-04-26T20:26:35.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.057+0000 7f264a7e4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f26440d0260 con 0x7f2644160390 2024-04-26T20:26:35.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.057+0000 7f264a7e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f26440d2270 con 0x7f26440d1950 2024-04-26T20:26:35.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.057+0000 7f2648d81640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2644162ee0 0x7f2644163370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48126/0 (socket says 172.21.15.37:48126) 2024-04-26T20:26:35.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.057+0000 7f2648d81640 1 -- 172.21.15.37:0/2228797668 learned_addr learned my addr 172.21.15.37:0/2228797668 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:35.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.057+0000 7f26437fe640 1 --2- 172.21.15.37:0/2228797668 >> v2:172.21.15.134:3300/0 conn(0x7f2644160390 0x7f26440d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.057+0000 7f2643fff640 1 --2- 172.21.15.37:0/2228797668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440d1950 0x7f26440cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.057+0000 7f2642ffd640 1 -- 172.21.15.37:0/2228797668 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 519097526 0 0) 0x7f264415d320 con 0x7f2644162ee0 2024-04-26T20:26:35.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.057+0000 7f2642ffd640 1 -- 172.21.15.37:0/2228797668 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2630003520 con 0x7f2644162ee0 2024-04-26T20:26:35.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.058+0000 7f26437fe640 1 -- 172.21.15.37:0/2228797668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440d1950 msgr2=0x7f26440cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.058+0000 7f26437fe640 1 --2- 172.21.15.37:0/2228797668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440d1950 0x7f26440cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.058+0000 7f26437fe640 1 -- 172.21.15.37:0/2228797668 >> v1:172.21.15.37:6789/0 conn(0x7f2644162ee0 legacy=0x7f2644163370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.058+0000 7f26437fe640 1 -- 172.21.15.37:0/2228797668 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26440d10c0 con 0x7f2644160390 2024-04-26T20:26:35.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.058+0000 7f2643fff640 1 --2- 172.21.15.37:0/2228797668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440d1950 0x7f26440cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:35.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.058+0000 7f26437fe640 1 --2- 172.21.15.37:0/2228797668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2644160390 0x7f26440d1d30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f262c0097a0 tx=0x7f262c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0cb65ebaff5bb1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.059+0000 7f2642ffd640 1 -- 172.21.15.37:0/2228797668 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c037070 con 0x7f2644160390 2024-04-26T20:26:35.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.059+0000 7f2642ffd640 1 -- 172.21.15.37:0/2228797668 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262c032440 con 0x7f2644160390 2024-04-26T20:26:35.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.059+0000 7f2642ffd640 1 -- 172.21.15.37:0/2228797668 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c032810 con 0x7f2644160390 2024-04-26T20:26:35.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.059+0000 7f264a7e4640 1 -- 172.21.15.37:0/2228797668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2644160390 msgr2=0x7f26440d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.059+0000 7f264a7e4640 1 --2- 172.21.15.37:0/2228797668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2644160390 0x7f26440d1d30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f262c0097a0 tx=0x7f262c02a180 comp rx=0 tx=0).stop 2024-04-26T20:26:35.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.059+0000 7f264a7e4640 1 -- 172.21.15.37:0/2228797668 shutdown_connections 2024-04-26T20:26:35.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.059+0000 7f264a7e4640 1 --2- 172.21.15.37:0/2228797668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2644160390 0x7f26440d1d30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.059+0000 7f264a7e4640 1 --2- 172.21.15.37:0/2228797668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440d1950 0x7f26440cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.059+0000 7f264a7e4640 1 -- 172.21.15.37:0/2228797668 >> 172.21.15.37:0/2228797668 conn(0x7f264409e4e0 msgr2=0x7f26440d5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.059+0000 7f264a7e4640 1 -- 172.21.15.37:0/2228797668 shutdown_connections 2024-04-26T20:26:35.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.059+0000 7f264a7e4640 1 -- 172.21.15.37:0/2228797668 wait complete. 2024-04-26T20:26:35.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.060+0000 7f264a7e4640 1 Processor -- start 2024-04-26T20:26:35.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.061+0000 7f264a7e4640 1 -- start start 2024-04-26T20:26:35.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.061+0000 7f264a7e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26440d1950 0x7f264415c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.061+0000 7f264a7e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2644160390 0x7f26440cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.061+0000 7f264a7e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2644162ee0 0x7f264415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.061+0000 7f264a7e4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26440d0a70 con 0x7f26440d1950 2024-04-26T20:26:35.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.061+0000 7f264a7e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26440d0790 con 0x7f2644162ee0 2024-04-26T20:26:35.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.061+0000 7f264a7e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f26440d0d50 con 0x7f2644160390 2024-04-26T20:26:35.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.061+0000 7f2643fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26440d1950 0x7f264415c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.061+0000 7f2643fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26440d1950 0x7f264415c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56016/0 (socket says 172.21.15.37:56016) 2024-04-26T20:26:35.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.061+0000 7f2643fff640 1 -- 172.21.15.37:0/3656303227 learned_addr learned my addr 172.21.15.37:0/3656303227 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:35.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f26437fe640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2644160390 0x7f26440cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f2648d81640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2644162ee0 0x7f264415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f2643fff640 1 -- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2644160390 msgr2=0x7f26440cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f2643fff640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2644160390 0x7f26440cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f2643fff640 1 -- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2644162ee0 msgr2=0x7f264415e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f2643fff640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2644162ee0 0x7f264415e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f2643fff640 1 -- 172.21.15.37:0/3656303227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26440b9b50 con 0x7f26440d1950 2024-04-26T20:26:35.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f2648d81640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2644162ee0 0x7f264415e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:35.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f2643fff640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26440d1950 0x7f264415c690 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f263400e960 tx=0x7f263400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f2640ff9640 1 -- 172.21.15.37:0/3656303227 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634017070 con 0x7f26440d1950 2024-04-26T20:26:35.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f2640ff9640 1 -- 172.21.15.37:0/3656303227 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263400c9a0 con 0x7f26440d1950 2024-04-26T20:26:35.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f264a7e4640 1 -- 172.21.15.37:0/3656303227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26440b9e30 con 0x7f26440d1950 2024-04-26T20:26:35.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f2640ff9640 1 -- 172.21.15.37:0/3656303227 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263400cd70 con 0x7f26440d1950 2024-04-26T20:26:35.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.062+0000 7f264a7e4640 1 -- 172.21.15.37:0/3656303227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26440ba3f0 con 0x7f26440d1950 2024-04-26T20:26:35.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.063+0000 7f264a7e4640 1 -- 172.21.15.37:0/3656303227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f26440cbfa0 con 0x7f26440d1950 2024-04-26T20:26:35.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.064+0000 7f2640ff9640 1 -- 172.21.15.37:0/3656303227 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f263401a070 con 0x7f26440d1950 2024-04-26T20:26:35.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.064+0000 7f2640ff9640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f262406cbc0 0x7f262406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.064+0000 7f2640ff9640 1 -- 172.21.15.37:0/3656303227 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f263408bb80 con 0x7f26440d1950 2024-04-26T20:26:35.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.064+0000 7f2640ff9640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] conn(0x7f2624072a80 0x7f2624074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.064+0000 7f2640ff9640 1 -- 172.21.15.37:0/3656303227 --> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2634022e50 con 0x7f2624072a80 2024-04-26T20:26:35.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.064+0000 7f2640ff9640 1 -- 172.21.15.37:0/3656303227 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2634092050 con 0x7f26440d1950 2024-04-26T20:26:35.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.064+0000 7f26437fe640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f262406cbc0 0x7f262406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.064+0000 7f2648d81640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] conn(0x7f2624072a80 0x7f2624074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.065+0000 7f2648d81640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] conn(0x7f2624072a80 0x7f2624074ea0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.065+0000 7f26437fe640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f262406cbc0 0x7f262406f080 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f262c034000 tx=0x7f262c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.065+0000 7f2640ff9640 1 -- 172.21.15.37:0/3656303227 <== osd.2 v2:172.21.15.37:6816/2890277493 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2634022e50 con 0x7f2624072a80 2024-04-26T20:26:35.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.078+0000 7f264a7e4640 1 -- 172.21.15.37:0/3656303227 --> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f26440d0080 con 0x7f2624072a80 2024-04-26T20:26:35.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.078+0000 7f2640ff9640 1 -- 172.21.15.37:0/3656303227 <== osd.2 v2:172.21.15.37:6816/2890277493 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f26440d0080 con 0x7f2624072a80 2024-04-26T20:26:35.080 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:26:35.080 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:26:35.080 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:26:35.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.079+0000 7f264a7e4640 1 -- 172.21.15.37:0/3656303227 >> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] conn(0x7f2624072a80 msgr2=0x7f2624074ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.079+0000 7f264a7e4640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.37:6816/2890277493,v1:172.21.15.37:6817/2890277493] conn(0x7f2624072a80 0x7f2624074ea0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.079+0000 7f264a7e4640 1 -- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f262406cbc0 msgr2=0x7f262406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.079+0000 7f264a7e4640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f262406cbc0 0x7f262406f080 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f262c034000 tx=0x7f262c02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:35.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.079+0000 7f264a7e4640 1 -- 172.21.15.37:0/3656303227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26440d1950 msgr2=0x7f264415c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.079+0000 7f264a7e4640 1 --2- 172.21.15.37:0/3656303227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26440d1950 0x7f264415c690 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f263400e960 tx=0x7f263400ee20 comp rx=0 tx=0).stop 2024-04-26T20:26:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.079+0000 7f2648d81640 1 -- 172.21.15.37:0/3656303227 reap_dead start 2024-04-26T20:26:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.079+0000 7f264a7e4640 1 -- 172.21.15.37:0/3656303227 shutdown_connections 2024-04-26T20:26:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.079+0000 7f264a7e4640 1 -- 172.21.15.37:0/3656303227 >> 172.21.15.37:0/3656303227 conn(0x7f264409e4e0 msgr2=0x7f26441632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.079+0000 7f264a7e4640 1 -- 172.21.15.37:0/3656303227 shutdown_connections 2024-04-26T20:26:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.079+0000 7f264a7e4640 1 -- 172.21.15.37:0/3656303227 wait complete. 2024-04-26T20:26:35.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T20:26:35.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.190+0000 7f3fc42f1640 1 Processor -- start 2024-04-26T20:26:35.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.190+0000 7f3fc42f1640 1 -- start start 2024-04-26T20:26:35.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.191+0000 7f3fc42f1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fbc162b40 0x7f3fbc0d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.191+0000 7f3fc42f1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3fbc0d3820 0x7f3fbc0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.191+0000 7f3fc42f1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3fbc0d1850 0x7f3fbc0d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.191+0000 7f3fc42f1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3fbc0d41c0 con 0x7f3fbc0d1850 2024-04-26T20:26:35.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.191+0000 7f3fc42f1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3fbc0d0210 con 0x7f3fbc0d3820 2024-04-26T20:26:35.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.191+0000 7f3fc42f1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3fbc0d04f0 con 0x7f3fbc162b40 2024-04-26T20:26:35.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.191+0000 7f3fc288e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3fbc0d1850 0x7f3fbc0d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.191+0000 7f3fc288e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3fbc0d1850 0x7f3fbc0d1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56018/0 (socket says 172.21.15.37:56018) 2024-04-26T20:26:35.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.191+0000 7f3fc288e640 1 -- 172.21.15.37:0/999239497 learned_addr learned my addr 172.21.15.37:0/999239497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.192+0000 7f3fc188c640 1 --2- 172.21.15.37:0/999239497 >> v2:172.21.15.134:3300/0 conn(0x7f3fbc0d3820 0x7f3fbc0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.192+0000 7f3fc208d640 1 --2- 172.21.15.37:0/999239497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fbc162b40 0x7f3fbc0d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.192+0000 7f3fc288e640 1 -- 172.21.15.37:0/999239497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fbc162b40 msgr2=0x7f3fbc0d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.192+0000 7f3fc288e640 1 --2- 172.21.15.37:0/999239497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fbc162b40 0x7f3fbc0d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.192+0000 7f3fc288e640 1 -- 172.21.15.37:0/999239497 >> v2:172.21.15.134:3300/0 conn(0x7f3fbc0d3820 msgr2=0x7f3fbc0d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.192+0000 7f3fc288e640 1 --2- 172.21.15.37:0/999239497 >> v2:172.21.15.134:3300/0 conn(0x7f3fbc0d3820 0x7f3fbc0d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.192+0000 7f3fc288e640 1 -- 172.21.15.37:0/999239497 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc0d07d0 con 0x7f3fbc0d1850 2024-04-26T20:26:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.192+0000 7f3fc288e640 1 --2- 172.21.15.37:0/999239497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fbc0d1850 0x7f3fbc0d1ce0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc06c690 tx=0x7f3fac02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e76b950cf27ce61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc108b640 1 -- 172.21.15.37:0/999239497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac036070 con 0x7f3fbc0d1850 2024-04-26T20:26:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc108b640 1 -- 172.21.15.37:0/999239497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fac031600 con 0x7f3fbc0d1850 2024-04-26T20:26:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc108b640 1 -- 172.21.15.37:0/999239497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac0319d0 con 0x7f3fbc0d1850 2024-04-26T20:26:35.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc42f1640 1 -- 172.21.15.37:0/999239497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fbc0d1850 msgr2=0x7f3fbc0d1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc42f1640 1 --2- 172.21.15.37:0/999239497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fbc0d1850 0x7f3fbc0d1ce0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc06c690 tx=0x7f3fac02a730 comp rx=0 tx=0).stop 2024-04-26T20:26:35.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc42f1640 1 -- 172.21.15.37:0/999239497 shutdown_connections 2024-04-26T20:26:35.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc42f1640 1 --2- 172.21.15.37:0/999239497 >> v2:172.21.15.134:3300/0 conn(0x7f3fbc0d3820 0x7f3fbc0d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc42f1640 1 --2- 172.21.15.37:0/999239497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fbc0d1850 0x7f3fbc0d1ce0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc42f1640 1 --2- 172.21.15.37:0/999239497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fbc162b40 0x7f3fbc0d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc42f1640 1 -- 172.21.15.37:0/999239497 >> 172.21.15.37:0/999239497 conn(0x7f3fbc09e550 msgr2=0x7f3fbc0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc42f1640 1 -- 172.21.15.37:0/999239497 shutdown_connections 2024-04-26T20:26:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.193+0000 7f3fc42f1640 1 -- 172.21.15.37:0/999239497 wait complete. 2024-04-26T20:26:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.194+0000 7f3fc42f1640 1 Processor -- start 2024-04-26T20:26:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.194+0000 7f3fc42f1640 1 -- start start 2024-04-26T20:26:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.195+0000 7f3fc42f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fbc0d1850 0x7f3fbc0c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.195+0000 7f3fc42f1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fbc0d3820 0x7f3fbc1f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.195+0000 7f3fc42f1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fbc162b40 0x7f3fbc1f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.195+0000 7f3fc42f1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc0d6110 con 0x7f3fbc162b40 2024-04-26T20:26:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.195+0000 7f3fc42f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc0d0d90 con 0x7f3fbc0d1850 2024-04-26T20:26:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.195+0000 7f3fc42f1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3fbc0d12b0 con 0x7f3fbc0d3820 2024-04-26T20:26:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.195+0000 7f3fc288e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fbc162b40 0x7f3fbc1f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.195+0000 7f3fc288e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fbc162b40 0x7f3fbc1f8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56024/0 (socket says 172.21.15.37:56024) 2024-04-26T20:26:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.195+0000 7f3fc288e640 1 -- 172.21.15.37:0/4602445 learned_addr learned my addr 172.21.15.37:0/4602445 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.195+0000 7f3fc188c640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fbc0d3820 0x7f3fbc1f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.195+0000 7f3fc208d640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fbc0d1850 0x7f3fbc0c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.196+0000 7f3fc288e640 1 -- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fbc0d3820 msgr2=0x7f3fbc1f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.196+0000 7f3fc288e640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fbc0d3820 0x7f3fbc1f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.196+0000 7f3fc288e640 1 -- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fbc0d1850 msgr2=0x7f3fbc0c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.196+0000 7f3fc288e640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fbc0d1850 0x7f3fbc0c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.196+0000 7f3fc288e640 1 -- 172.21.15.37:0/4602445 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc0b27c0 con 0x7f3fbc162b40 2024-04-26T20:26:35.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.196+0000 7f3fc288e640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fbc162b40 0x7f3fbc1f8e20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3fac034040 tx=0x7f3fac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.196+0000 7f3fbaffd640 1 -- 172.21.15.37:0/4602445 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac036050 con 0x7f3fbc162b40 2024-04-26T20:26:35.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.196+0000 7f3fbaffd640 1 -- 172.21.15.37:0/4602445 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fac042070 con 0x7f3fbc162b40 2024-04-26T20:26:35.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.196+0000 7f3fbaffd640 1 -- 172.21.15.37:0/4602445 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac009510 con 0x7f3fbc162b40 2024-04-26T20:26:35.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.196+0000 7f3fc42f1640 1 -- 172.21.15.37:0/4602445 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fbc0b2aa0 con 0x7f3fbc162b40 2024-04-26T20:26:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.196+0000 7f3fc42f1640 1 -- 172.21.15.37:0/4602445 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fbc0b3060 con 0x7f3fbc162b40 2024-04-26T20:26:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.197+0000 7f3fc42f1640 1 -- 172.21.15.37:0/4602445 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fbc06a360 con 0x7f3fbc162b40 2024-04-26T20:26:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.197+0000 7f3fbaffd640 1 -- 172.21.15.37:0/4602445 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f3fac007bf0 con 0x7f3fbc162b40 2024-04-26T20:26:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.198+0000 7f3fbaffd640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3fa806cbf0 0x7f3fa806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.198+0000 7f3fbaffd640 1 -- 172.21.15.37:0/4602445 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f3fac0aca50 con 0x7f3fbc162b40 2024-04-26T20:26:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.198+0000 7f3fc208d640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3fa806cbf0 0x7f3fa806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.199+0000 7f3fc208d640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3fa806cbf0 0x7f3fa806f0b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4006fd0 tx=0x7f3fb4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.203+0000 7f3fbaffd640 1 -- 172.21.15.37:0/4602445 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3fac07a6d0 con 0x7f3fbc162b40 2024-04-26T20:26:35.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.387+0000 7f3fc42f1640 1 -- 172.21.15.37:0/4602445 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f3fbc0d2960 con 0x7f3fa806cbf0 2024-04-26T20:26:35.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.387+0000 7f3fbaffd640 1 -- 172.21.15.37:0/4602445 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3fbc0d2960 con 0x7f3fa806cbf0 2024-04-26T20:26:35.389 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T20:26:35.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.389+0000 7f3fc42f1640 1 -- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3fa806cbf0 msgr2=0x7f3fa806f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.389+0000 7f3fc42f1640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3fa806cbf0 0x7f3fa806f0b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4006fd0 tx=0x7f3fb4008040 comp rx=0 tx=0).stop 2024-04-26T20:26:35.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.389+0000 7f3fc42f1640 1 -- 172.21.15.37:0/4602445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fbc162b40 msgr2=0x7f3fbc1f8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.389+0000 7f3fc42f1640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fbc162b40 0x7f3fbc1f8e20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3fac034040 tx=0x7f3fac004a40 comp rx=0 tx=0).stop 2024-04-26T20:26:35.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.390+0000 7f3fc42f1640 1 -- 172.21.15.37:0/4602445 shutdown_connections 2024-04-26T20:26:35.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.390+0000 7f3fc42f1640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fbc162b40 0x7f3fbc1f8e20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.390+0000 7f3fc42f1640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f3fa806cbf0 0x7f3fa806f0b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.390+0000 7f3fc42f1640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fbc0d3820 0x7f3fbc1f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.390+0000 7f3fc42f1640 1 --2- 172.21.15.37:0/4602445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fbc0d1850 0x7f3fbc0c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.390+0000 7f3fc42f1640 1 -- 172.21.15.37:0/4602445 >> 172.21.15.37:0/4602445 conn(0x7f3fbc09e550 msgr2=0x7f3fbc160130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.390+0000 7f3fc42f1640 1 -- 172.21.15.37:0/4602445 shutdown_connections 2024-04-26T20:26:35.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.390+0000 7f3fc42f1640 1 -- 172.21.15.37:0/4602445 wait complete. 2024-04-26T20:26:35.401 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T20:26:35.401 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:26:35.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.506+0000 7fec2acf2640 1 Processor -- start 2024-04-26T20:26:35.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.506+0000 7fec2acf2640 1 -- start start 2024-04-26T20:26:35.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec2acf2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec24160390 0x7fec240d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec2acf2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec2415d320 con 0x7fec240d1950 2024-04-26T20:26:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec2acf2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec240d10c0 con 0x7fec240cfd20 2024-04-26T20:26:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec2acf2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec240d04b0 con 0x7fec24160390 2024-04-26T20:26:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec28a8e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fec240d1950 0x7fec240d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48132/0 (socket says 172.21.15.37:48132) 2024-04-26T20:26:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec28a8e640 1 -- 172.21.15.37:0/3705848628 learned_addr learned my addr 172.21.15.37:0/3705848628 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec2928f640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec24160390 0x7fec240d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2530511439 0 0) 0x7fec2415d320 con 0x7fec240d1950 2024-04-26T20:26:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec14003520 con 0x7fec240d1950 2024-04-26T20:26:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3031835069 0 0) 0x7fec240d10c0 con 0x7fec240cfd20 2024-04-26T20:26:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec2415d320 con 0x7fec240cfd20 2024-04-26T20:26:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3835438053 0 0) 0x7fec14003520 con 0x7fec240d1950 2024-04-26T20:26:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec24160390 msgr2=0x7fec240d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec237fe640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec24160390 0x7fec240d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 >> v1:172.21.15.134:6789/0 conn(0x7fec240cfd20 legacy=0x7fec240d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec240d0790 con 0x7fec240d1950 2024-04-26T20:26:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.507+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7fec18001670 con 0x7fec240d1950 2024-04-26T20:26:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 >> v1:172.21.15.37:6789/0 conn(0x7fec240d1950 legacy=0x7fec240d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec237fe640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec24160390 0x7fec14003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec237fe640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec140040d0 0x7fec14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec237fe640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec14004d40 0x7fec140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec240d04b0 con 0x7fec140040d0 2024-04-26T20:26:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec1400baa0 con 0x7fec14004d40 2024-04-26T20:26:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec1400bca0 con 0x7fec24160390 2024-04-26T20:26:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec23fff640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec140040d0 0x7fec14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec28a8e640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec24160390 0x7fec14003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec2928f640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec14004d40 0x7fec140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec23fff640 1 -- 172.21.15.37:0/3705848628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec24160390 msgr2=0x7fec14003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec23fff640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec24160390 0x7fec14003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec23fff640 1 -- 172.21.15.37:0/3705848628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec14004d40 msgr2=0x7fec140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec23fff640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec14004d40 0x7fec140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.508+0000 7fec23fff640 1 -- 172.21.15.37:0/3705848628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec14003520 con 0x7fec140040d0 2024-04-26T20:26:35.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.509+0000 7fec23fff640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec140040d0 0x7fec14004800 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fec100018d0 tx=0x7fec14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6073af74943b005b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.509+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec100014f0 con 0x7fec140040d0 2024-04-26T20:26:35.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.509+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec100052d0 con 0x7fec140040d0 2024-04-26T20:26:35.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.509+0000 7fec237fe640 1 -- 172.21.15.37:0/3705848628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1002aca0 con 0x7fec140040d0 2024-04-26T20:26:35.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.509+0000 7fec2acf2640 1 -- 172.21.15.37:0/3705848628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec140040d0 msgr2=0x7fec14004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.509+0000 7fec2acf2640 1 --2- 172.21.15.37:0/3705848628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec140040d0 0x7fec14004800 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fec100018d0 tx=0x7fec14005830 comp rx=0 tx=0).stop 2024-04-26T20:26:35.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.509+0000 7fec2928f640 1 -- 172.21.15.37:0/3705848628 reap_dead start 2024-04-26T20:26:35.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.509+0000 7fec2acf2640 1 -- 172.21.15.37:0/3705848628 shutdown_connections 2024-04-26T20:26:35.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.509+0000 7fec2acf2640 1 -- 172.21.15.37:0/3705848628 >> 172.21.15.37:0/3705848628 conn(0x7fec2409e4e0 msgr2=0x7fec240d5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.510+0000 7fec2acf2640 1 -- 172.21.15.37:0/3705848628 shutdown_connections 2024-04-26T20:26:35.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.510+0000 7fec2acf2640 1 -- 172.21.15.37:0/3705848628 wait complete. 2024-04-26T20:26:35.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.511+0000 7fec2acf2640 1 Processor -- start 2024-04-26T20:26:35.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.511+0000 7fec2acf2640 1 -- start start 2024-04-26T20:26:35.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.511+0000 7fec2acf2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec140040d0 0x7fec240cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec2acf2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec240cefc0 0x7fec240cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec2acf2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec240cc740 0x7fec240ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec2acf2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec240d01b0 con 0x7fec140040d0 2024-04-26T20:26:35.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec2acf2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec240d0d50 con 0x7fec240cefc0 2024-04-26T20:26:35.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec2acf2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec24160820 con 0x7fec240cc740 2024-04-26T20:26:35.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec28a8e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec140040d0 0x7fec240cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec28a8e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec140040d0 0x7fec240cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56046/0 (socket says 172.21.15.37:56046) 2024-04-26T20:26:35.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec28a8e640 1 -- 172.21.15.37:0/2693595812 learned_addr learned my addr 172.21.15.37:0/2693595812 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:35.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec28a8e640 1 -- 172.21.15.37:0/2693595812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec240cc740 msgr2=0x7fec240ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec28a8e640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec240cc740 0x7fec240ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec28a8e640 1 -- 172.21.15.37:0/2693595812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec240cefc0 msgr2=0x7fec240cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec28a8e640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec240cefc0 0x7fec240cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec28a8e640 1 -- 172.21.15.37:0/2693595812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec241629c0 con 0x7fec140040d0 2024-04-26T20:26:35.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec28a8e640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec140040d0 0x7fec240cea80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fec18007370 tx=0x7fec18007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec217fa640 1 -- 172.21.15.37:0/2693595812 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1800f940 con 0x7fec140040d0 2024-04-26T20:26:35.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec217fa640 1 -- 172.21.15.37:0/2693595812 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec18018070 con 0x7fec140040d0 2024-04-26T20:26:35.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec2acf2640 1 -- 172.21.15.37:0/2693595812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec24162c60 con 0x7fec140040d0 2024-04-26T20:26:35.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec217fa640 1 -- 172.21.15.37:0/2693595812 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1800fd60 con 0x7fec140040d0 2024-04-26T20:26:35.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.512+0000 7fec2acf2640 1 -- 172.21.15.37:0/2693595812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec24163220 con 0x7fec140040d0 2024-04-26T20:26:35.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.513+0000 7fec2acf2640 1 -- 172.21.15.37:0/2693595812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec2406a360 con 0x7fec140040d0 2024-04-26T20:26:35.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.514+0000 7fec217fa640 1 -- 172.21.15.37:0/2693595812 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fec1801d460 con 0x7fec140040d0 2024-04-26T20:26:35.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.514+0000 7fec217fa640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fec0806cbc0 0x7fec0806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.514+0000 7fec217fa640 1 -- 172.21.15.37:0/2693595812 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fec1808bd00 con 0x7fec140040d0 2024-04-26T20:26:35.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.514+0000 7fec217fa640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] conn(0x7fec08072a80 0x7fec08074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.514+0000 7fec217fa640 1 -- 172.21.15.37:0/2693595812 --> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec08075550 con 0x7fec08072a80 2024-04-26T20:26:35.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.514+0000 7fec217fa640 1 -- 172.21.15.37:0/2693595812 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fec18090030 con 0x7fec140040d0 2024-04-26T20:26:35.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.514+0000 7fec23fff640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fec0806cbc0 0x7fec0806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.514+0000 7fec2928f640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] conn(0x7fec08072a80 0x7fec08074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.515+0000 7fec2928f640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] conn(0x7fec08072a80 0x7fec08074ea0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.515+0000 7fec23fff640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fec0806cbc0 0x7fec0806f080 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fec10032000 tx=0x7fec1002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.515+0000 7fec217fa640 1 -- 172.21.15.37:0/2693595812 <== osd.3 v2:172.21.15.37:6819/2156822987 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec08075550 con 0x7fec08072a80 2024-04-26T20:26:35.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.528+0000 7fec2acf2640 1 -- 172.21.15.37:0/2693595812 --> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fec240d2850 con 0x7fec08072a80 2024-04-26T20:26:35.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec217fa640 1 -- 172.21.15.37:0/2693595812 <== osd.3 v2:172.21.15.37:6819/2156822987 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fec240d2850 con 0x7fec08072a80 2024-04-26T20:26:35.530 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:26:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:26:35.530 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:26:35.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec2acf2640 1 -- 172.21.15.37:0/2693595812 >> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] conn(0x7fec08072a80 msgr2=0x7fec08074ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec2acf2640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.37:6819/2156822987,v1:172.21.15.37:6822/2156822987] conn(0x7fec08072a80 0x7fec08074ea0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec2acf2640 1 -- 172.21.15.37:0/2693595812 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fec0806cbc0 msgr2=0x7fec0806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec2acf2640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fec0806cbc0 0x7fec0806f080 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fec10032000 tx=0x7fec1002e000 comp rx=0 tx=0).stop 2024-04-26T20:26:35.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec2acf2640 1 -- 172.21.15.37:0/2693595812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec140040d0 msgr2=0x7fec240cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec2acf2640 1 --2- 172.21.15.37:0/2693595812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec140040d0 0x7fec240cea80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fec18007370 tx=0x7fec18007830 comp rx=0 tx=0).stop 2024-04-26T20:26:35.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec2928f640 1 -- 172.21.15.37:0/2693595812 reap_dead start 2024-04-26T20:26:35.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec2acf2640 1 -- 172.21.15.37:0/2693595812 shutdown_connections 2024-04-26T20:26:35.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec2acf2640 1 -- 172.21.15.37:0/2693595812 >> 172.21.15.37:0/2693595812 conn(0x7fec2409e4e0 msgr2=0x7fec240d1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec2acf2640 1 -- 172.21.15.37:0/2693595812 shutdown_connections 2024-04-26T20:26:35.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.529+0000 7fec2acf2640 1 -- 172.21.15.37:0/2693595812 wait complete. 2024-04-26T20:26:35.540 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T20:26:35.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.641+0000 7f03c5be9640 1 Processor -- start 2024-04-26T20:26:35.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.641+0000 7f03c5be9640 1 -- start start 2024-04-26T20:26:35.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03c5be9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c0159a90 0x7f03c006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03c5be9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f03c0070130 0x7f03c015fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03c5be9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f03c0160150 0x7f03c015c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03c5be9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f03c0160580 con 0x7f03c0070130 2024-04-26T20:26:35.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03c5be9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f03c0072e30 con 0x7f03c0160150 2024-04-26T20:26:35.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03c5be9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f03c0073110 con 0x7f03c0159a90 2024-04-26T20:26:35.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03beffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f03c0070130 0x7f03c015fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03beffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f03c0070130 0x7f03c015fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56048/0 (socket says 172.21.15.37:56048) 2024-04-26T20:26:35.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03beffd640 1 -- 172.21.15.37:0/3780962642 learned_addr learned my addr 172.21.15.37:0/3780962642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:35.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03bffff640 1 --2- 172.21.15.37:0/3780962642 >> v2:172.21.15.134:3300/0 conn(0x7f03c0160150 0x7f03c015c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03bf7fe640 1 --2- 172.21.15.37:0/3780962642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c0159a90 0x7f03c006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03beffd640 1 -- 172.21.15.37:0/3780962642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c0159a90 msgr2=0x7f03c006fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03beffd640 1 --2- 172.21.15.37:0/3780962642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c0159a90 0x7f03c006fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03beffd640 1 -- 172.21.15.37:0/3780962642 >> v2:172.21.15.134:3300/0 conn(0x7f03c0160150 msgr2=0x7f03c015c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03beffd640 1 --2- 172.21.15.37:0/3780962642 >> v2:172.21.15.134:3300/0 conn(0x7f03c0160150 0x7f03c015c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.642+0000 7f03beffd640 1 -- 172.21.15.37:0/3780962642 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c00733f0 con 0x7f03c0070130 2024-04-26T20:26:35.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.643+0000 7f03beffd640 1 --2- 172.21.15.37:0/3780962642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03c0070130 0x7f03c015fc10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f03c0073040 tx=0x7f03b402a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb98926fed3690c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.643+0000 7f03be7fc640 1 -- 172.21.15.37:0/3780962642 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4037070 con 0x7f03c0070130 2024-04-26T20:26:35.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.643+0000 7f03be7fc640 1 -- 172.21.15.37:0/3780962642 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b402adf0 con 0x7f03c0070130 2024-04-26T20:26:35.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.643+0000 7f03be7fc640 1 -- 172.21.15.37:0/3780962642 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4032660 con 0x7f03c0070130 2024-04-26T20:26:35.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.644+0000 7f03c5be9640 1 -- 172.21.15.37:0/3780962642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03c0070130 msgr2=0x7f03c015fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.644+0000 7f03c5be9640 1 --2- 172.21.15.37:0/3780962642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03c0070130 0x7f03c015fc10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f03c0073040 tx=0x7f03b402a0a0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.644+0000 7f03c5be9640 1 -- 172.21.15.37:0/3780962642 shutdown_connections 2024-04-26T20:26:35.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.644+0000 7f03c5be9640 1 --2- 172.21.15.37:0/3780962642 >> v2:172.21.15.134:3300/0 conn(0x7f03c0160150 0x7f03c015c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.644+0000 7f03c5be9640 1 --2- 172.21.15.37:0/3780962642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03c0070130 0x7f03c015fc10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.644+0000 7f03c5be9640 1 --2- 172.21.15.37:0/3780962642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c0159a90 0x7f03c006fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.644+0000 7f03c5be9640 1 -- 172.21.15.37:0/3780962642 >> 172.21.15.37:0/3780962642 conn(0x7f03c00d0d50 msgr2=0x7f03c00d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.644+0000 7f03c5be9640 1 -- 172.21.15.37:0/3780962642 shutdown_connections 2024-04-26T20:26:35.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.644+0000 7f03c5be9640 1 -- 172.21.15.37:0/3780962642 wait complete. 2024-04-26T20:26:35.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.645+0000 7f03c5be9640 1 Processor -- start 2024-04-26T20:26:35.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.645+0000 7f03c5be9640 1 -- start start 2024-04-26T20:26:35.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03c5be9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0070130 0x7f03c01f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03c5be9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03c0159a90 0x7f03c014f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03c5be9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c0160150 0x7f03c014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03c5be9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0159e70 con 0x7f03c0159a90 2024-04-26T20:26:35.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03c5be9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01609d0 con 0x7f03c0070130 2024-04-26T20:26:35.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03c5be9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f03c0159ff0 con 0x7f03c0160150 2024-04-26T20:26:35.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03beffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03c0159a90 0x7f03c014f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03beffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03c0159a90 0x7f03c014f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56064/0 (socket says 172.21.15.37:56064) 2024-04-26T20:26:35.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03beffd640 1 -- 172.21.15.37:0/2252663326 learned_addr learned my addr 172.21.15.37:0/2252663326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:35.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03bffff640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c0160150 0x7f03c014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03bf7fe640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0070130 0x7f03c01f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03beffd640 1 -- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c0160150 msgr2=0x7f03c014fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03beffd640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c0160150 0x7f03c014fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03beffd640 1 -- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0070130 msgr2=0x7f03c01f9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03beffd640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0070130 0x7f03c01f9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03beffd640 1 -- 172.21.15.37:0/2252663326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c00b4890 con 0x7f03c0159a90 2024-04-26T20:26:35.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03bffff640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c0160150 0x7f03c014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03bf7fe640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0070130 0x7f03c01f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.646+0000 7f03beffd640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03c0159a90 0x7f03c014f590 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f03b402a0e0 tx=0x7f03b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.647+0000 7f03a7fff640 1 -- 172.21.15.37:0/2252663326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4041020 con 0x7f03c0159a90 2024-04-26T20:26:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.647+0000 7f03a7fff640 1 -- 172.21.15.37:0/2252663326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b403b610 con 0x7f03c0159a90 2024-04-26T20:26:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.647+0000 7f03a7fff640 1 -- 172.21.15.37:0/2252663326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4037040 con 0x7f03c0159a90 2024-04-26T20:26:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.647+0000 7f03c5be9640 1 -- 172.21.15.37:0/2252663326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c00b4b30 con 0x7f03c0159a90 2024-04-26T20:26:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.647+0000 7f03c5be9640 1 -- 172.21.15.37:0/2252663326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c00b50f0 con 0x7f03c0159a90 2024-04-26T20:26:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.648+0000 7f03c5be9640 1 -- 172.21.15.37:0/2252663326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c006a360 con 0x7f03c0159a90 2024-04-26T20:26:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.648+0000 7f03a7fff640 1 -- 172.21.15.37:0/2252663326 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f03b4044020 con 0x7f03c0159a90 2024-04-26T20:26:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.648+0000 7f03a7fff640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03ac06cbf0 0x7f03ac06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.649+0000 7f03a7fff640 1 -- 172.21.15.37:0/2252663326 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f03b40abda0 con 0x7f03c0159a90 2024-04-26T20:26:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.649+0000 7f03bf7fe640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03ac06cbf0 0x7f03ac06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.649+0000 7f03bf7fe640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03ac06cbf0 0x7f03ac06f0b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f03b0006fd0 tx=0x7f03b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.653+0000 7f03a7fff640 1 -- 172.21.15.37:0/2252663326 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03b404b030 con 0x7f03c0159a90 2024-04-26T20:26:35.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.840+0000 7f03c5be9640 1 -- 172.21.15.37:0/2252663326 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f03c00d0730 con 0x7f03ac06cbf0 2024-04-26T20:26:35.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.841+0000 7f03a7fff640 1 -- 172.21.15.37:0/2252663326 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f03c00d0730 con 0x7f03ac06cbf0 2024-04-26T20:26:35.842 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T20:26:35.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 -- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03ac06cbf0 msgr2=0x7f03ac06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03ac06cbf0 0x7f03ac06f0b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f03b0006fd0 tx=0x7f03b0008040 comp rx=0 tx=0).stop 2024-04-26T20:26:35.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 -- 172.21.15.37:0/2252663326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03c0159a90 msgr2=0x7f03c014f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03c0159a90 0x7f03c014f590 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f03b402a0e0 tx=0x7f03b40046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 -- 172.21.15.37:0/2252663326 shutdown_connections 2024-04-26T20:26:35.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f03ac06cbf0 0x7f03ac06f0b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c0160150 0x7f03c014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03c0159a90 0x7f03c014f590 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 --2- 172.21.15.37:0/2252663326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0070130 0x7f03c01f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 -- 172.21.15.37:0/2252663326 >> 172.21.15.37:0/2252663326 conn(0x7f03c00d0d50 msgr2=0x7f03c00d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 -- 172.21.15.37:0/2252663326 shutdown_connections 2024-04-26T20:26:35.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.843+0000 7f03c5be9640 1 -- 172.21.15.37:0/2252663326 wait complete. 2024-04-26T20:26:35.855 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T20:26:35.855 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:26:35.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.958+0000 7f51d759e640 1 Processor -- start 2024-04-26T20:26:35.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.959+0000 7f51d759e640 1 -- start start 2024-04-26T20:26:35.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.959+0000 7f51d759e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f51d015f5d0 0x7f51d00d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.959+0000 7f51d759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00d4010 0x7f51d00d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.959+0000 7f51d759e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51d00d5920 con 0x7f51d00d5470 2024-04-26T20:26:35.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.959+0000 7f51d759e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f51d00b5420 con 0x7f51d015f5d0 2024-04-26T20:26:35.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.959+0000 7f51d759e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51d00b50f0 con 0x7f51d00d4010 2024-04-26T20:26:35.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51d5b3b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f51d00d5470 0x7f51d00d1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:35084/0 (socket says 172.21.15.37:35084) 2024-04-26T20:26:35.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51d5b3b640 1 -- 172.21.15.37:0/382022658 learned_addr learned my addr 172.21.15.37:0/382022658 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:35.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1824448248 0 0) 0x7f51c8001490 con 0x7f51d00d5470 2024-04-26T20:26:35.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51d4b39640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00d4010 0x7f51d00d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51d533a640 1 --2- 172.21.15.37:0/382022658 >> v2:172.21.15.134:3300/0 conn(0x7f51d015f5d0 0x7f51d00d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51bc003520 con 0x7f51d00d5470 2024-04-26T20:26:35.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4020178956 0 0) 0x7f51bc003520 con 0x7f51d00d5470 2024-04-26T20:26:35.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00d4010 msgr2=0x7f51d00d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00d4010 0x7f51d00d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 >> v2:172.21.15.134:3300/0 conn(0x7f51d015f5d0 msgr2=0x7f51d00d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 --2- 172.21.15.37:0/382022658 >> v2:172.21.15.134:3300/0 conn(0x7f51d015f5d0 0x7f51d00d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d00d0a80 con 0x7f51d00d5470 2024-04-26T20:26:35.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f51c8001bc0 con 0x7f51d00d5470 2024-04-26T20:26:35.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 >> v1:172.21.15.37:6789/0 conn(0x7f51d00d5470 legacy=0x7f51d00d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc0039b0 0x7f51bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51bc004800 0x7f51bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00d4010 0x7f51bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f51d00b5420 con 0x7f51bc0039b0 2024-04-26T20:26:35.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51d00b50f0 con 0x7f51bc004800 2024-04-26T20:26:35.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51bc00b9f0 con 0x7f51d00d4010 2024-04-26T20:26:35.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.960+0000 7f51d533a640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc0039b0 0x7f51bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51d4b39640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51bc004800 0x7f51bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51d5b3b640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00d4010 0x7f51bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51d533a640 1 -- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00d4010 msgr2=0x7f51bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51d533a640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00d4010 0x7f51bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51d533a640 1 -- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51bc004800 msgr2=0x7f51bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51d533a640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51bc004800 0x7f51bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51d533a640 1 -- 172.21.15.37:0/382022658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc003520 con 0x7f51bc0039b0 2024-04-26T20:26:35.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51d4b39640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51bc004800 0x7f51bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:35.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51d533a640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc0039b0 0x7f51bc003dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009ac0 tx=0x7f51bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b090b57796b35d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc036070 con 0x7f51bc0039b0 2024-04-26T20:26:35.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51cc031400 con 0x7f51bc0039b0 2024-04-26T20:26:35.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.961+0000 7f51c7fff640 1 -- 172.21.15.37:0/382022658 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc009690 con 0x7f51bc0039b0 2024-04-26T20:26:35.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.962+0000 7f51d759e640 1 -- 172.21.15.37:0/382022658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc0039b0 msgr2=0x7f51bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.962+0000 7f51d759e640 1 --2- 172.21.15.37:0/382022658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc0039b0 0x7f51bc003dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009ac0 tx=0x7f51bc005830 comp rx=0 tx=0).stop 2024-04-26T20:26:35.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.962+0000 7f51d5b3b640 1 -- 172.21.15.37:0/382022658 reap_dead start 2024-04-26T20:26:35.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.962+0000 7f51d759e640 1 -- 172.21.15.37:0/382022658 shutdown_connections 2024-04-26T20:26:35.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.962+0000 7f51d759e640 1 -- 172.21.15.37:0/382022658 >> 172.21.15.37:0/382022658 conn(0x7f51d008ebd0 msgr2=0x7f51d009e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.962+0000 7f51d759e640 1 -- 172.21.15.37:0/382022658 shutdown_connections 2024-04-26T20:26:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.962+0000 7f51d759e640 1 -- 172.21.15.37:0/382022658 wait complete. 2024-04-26T20:26:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.963+0000 7f51d759e640 1 Processor -- start 2024-04-26T20:26:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.963+0000 7f51d759e640 1 -- start start 2024-04-26T20:26:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.964+0000 7f51d759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51bc0039b0 0x7f51d01f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.964+0000 7f51d759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00cc290 0x7f51d00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.964+0000 7f51d759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d00cff50 0x7f51d014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.964+0000 7f51d759e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f51d015f9b0 con 0x7f51d00cff50 2024-04-26T20:26:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.964+0000 7f51d759e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51d00d1040 con 0x7f51bc0039b0 2024-04-26T20:26:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.964+0000 7f51d759e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51d00d34f0 con 0x7f51d00cc290 2024-04-26T20:26:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.964+0000 7f51d5b3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d00cff50 0x7f51d014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.964+0000 7f51d5b3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d00cff50 0x7f51d014f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41948/0 (socket says 172.21.15.37:41948) 2024-04-26T20:26:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.964+0000 7f51d5b3b640 1 -- 172.21.15.37:0/107934798 learned_addr learned my addr 172.21.15.37:0/107934798 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.964+0000 7f51d533a640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51bc0039b0 0x7f51d01f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.964+0000 7f51d4b39640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00cc290 0x7f51d00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.965+0000 7f51d5b3b640 1 -- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00cc290 msgr2=0x7f51d00cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.965+0000 7f51d5b3b640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51d00cc290 0x7f51d00cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.965+0000 7f51d5b3b640 1 -- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51bc0039b0 msgr2=0x7f51d01f93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.965+0000 7f51d5b3b640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51bc0039b0 0x7f51d01f93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.965+0000 7f51d5b3b640 1 -- 172.21.15.37:0/107934798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d00cdf00 con 0x7f51d00cff50 2024-04-26T20:26:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.965+0000 7f51d5b3b640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d00cff50 0x7f51d014f1f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f51c8002230 tx=0x7f51c800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.965+0000 7f51c5ffb640 1 -- 172.21.15.37:0/107934798 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8019070 con 0x7f51d00cff50 2024-04-26T20:26:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.965+0000 7f51c5ffb640 1 -- 172.21.15.37:0/107934798 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c800eb90 con 0x7f51d00cff50 2024-04-26T20:26:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.965+0000 7f51d759e640 1 -- 172.21.15.37:0/107934798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d00ce1e0 con 0x7f51d00cff50 2024-04-26T20:26:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.965+0000 7f51c5ffb640 1 -- 172.21.15.37:0/107934798 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8014480 con 0x7f51d00cff50 2024-04-26T20:26:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.965+0000 7f51d759e640 1 -- 172.21.15.37:0/107934798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d00ce7a0 con 0x7f51d00cff50 2024-04-26T20:26:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.966+0000 7f51d759e640 1 -- 172.21.15.37:0/107934798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51d006a360 con 0x7f51d00cff50 2024-04-26T20:26:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.966+0000 7f51c5ffb640 1 -- 172.21.15.37:0/107934798 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f51c801c070 con 0x7f51d00cff50 2024-04-26T20:26:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.967+0000 7f51c5ffb640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f51b406cbc0 0x7f51b406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.967+0000 7f51c5ffb640 1 -- 172.21.15.37:0/107934798 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f51c808b8b0 con 0x7f51d00cff50 2024-04-26T20:26:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.967+0000 7f51c5ffb640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] conn(0x7f51b4072a80 0x7f51b4074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.967+0000 7f51c5ffb640 1 -- 172.21.15.37:0/107934798 --> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51c800ce90 con 0x7f51b4072a80 2024-04-26T20:26:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.967+0000 7f51d533a640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f51b406cbc0 0x7f51b406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.967+0000 7f51c5ffb640 1 -- 172.21.15.37:0/107934798 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f51c80902a0 con 0x7f51d00cff50 2024-04-26T20:26:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.967+0000 7f51d4b39640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] conn(0x7f51b4072a80 0x7f51b4074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.968+0000 7f51d533a640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f51b406cbc0 0x7f51b406f080 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f51cc033000 tx=0x7f51cc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.968+0000 7f51d4b39640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] conn(0x7f51b4072a80 0x7f51b4074ea0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.968+0000 7f51c5ffb640 1 -- 172.21.15.37:0/107934798 <== osd.4 v2:172.21.15.134:6824/2837585248 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51c800ce90 con 0x7f51b4072a80 2024-04-26T20:26:35.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.981+0000 7f51d759e640 1 -- 172.21.15.37:0/107934798 --> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f51d00b4880 con 0x7f51b4072a80 2024-04-26T20:26:35.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.982+0000 7f51c5ffb640 1 -- 172.21.15.37:0/107934798 <== osd.4 v2:172.21.15.134:6824/2837585248 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f51d00b4880 con 0x7f51b4072a80 2024-04-26T20:26:35.983 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:26:35.983 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:26:35.983 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:26:35.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.982+0000 7f51d759e640 1 -- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] conn(0x7f51b4072a80 msgr2=0x7f51b4074ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.982+0000 7f51d759e640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:6824/2837585248,v1:172.21.15.134:6825/2837585248] conn(0x7f51b4072a80 0x7f51b4074ea0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.982+0000 7f51d759e640 1 -- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f51b406cbc0 msgr2=0x7f51b406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.982+0000 7f51d759e640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f51b406cbc0 0x7f51b406f080 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f51cc033000 tx=0x7f51cc008030 comp rx=0 tx=0).stop 2024-04-26T20:26:35.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.982+0000 7f51d759e640 1 -- 172.21.15.37:0/107934798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d00cff50 msgr2=0x7f51d014f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.982+0000 7f51d759e640 1 --2- 172.21.15.37:0/107934798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d00cff50 0x7f51d014f1f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f51c8002230 tx=0x7f51c800e530 comp rx=0 tx=0).stop 2024-04-26T20:26:35.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.982+0000 7f51d5b3b640 1 -- 172.21.15.37:0/107934798 reap_dead start 2024-04-26T20:26:35.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.982+0000 7f51d759e640 1 -- 172.21.15.37:0/107934798 shutdown_connections 2024-04-26T20:26:35.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.982+0000 7f51d759e640 1 -- 172.21.15.37:0/107934798 >> 172.21.15.37:0/107934798 conn(0x7f51d008ebd0 msgr2=0x7f51d00b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.982+0000 7f51d759e640 1 -- 172.21.15.37:0/107934798 shutdown_connections 2024-04-26T20:26:35.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:35.983+0000 7f51d759e640 1 -- 172.21.15.37:0/107934798 wait complete. 2024-04-26T20:26:35.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T20:26:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.094+0000 7fee0337f640 1 Processor -- start 2024-04-26T20:26:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.094+0000 7fee0337f640 1 -- start start 2024-04-26T20:26:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.095+0000 7fee0337f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedfc15f380 0x7fedfc0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.095+0000 7fee0337f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fedfc15f760 0x7fedfc0d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.095+0000 7fee0337f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fedfc0d5b50 0x7fedfc0cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.095+0000 7fee0337f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fedfc0d43a0 con 0x7fedfc15f760 2024-04-26T20:26:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.095+0000 7fee0337f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fedfc0d3de0 con 0x7fedfc0d5b50 2024-04-26T20:26:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.095+0000 7fee0337f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fedfc0d40c0 con 0x7fedfc15f380 2024-04-26T20:26:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.095+0000 7fee0091a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fedfc15f760 0x7fedfc0d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.095+0000 7fee0091a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fedfc15f760 0x7fedfc0d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41958/0 (socket says 172.21.15.37:41958) 2024-04-26T20:26:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.095+0000 7fee0091a640 1 -- 172.21.15.37:0/66380304 learned_addr learned my addr 172.21.15.37:0/66380304 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.095+0000 7fee0191c640 1 --2- 172.21.15.37:0/66380304 >> v2:172.21.15.134:3300/0 conn(0x7fedfc0d5b50 0x7fedfc0cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.095+0000 7fee0111b640 1 --2- 172.21.15.37:0/66380304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedfc15f380 0x7fedfc0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.096+0000 7fee0091a640 1 -- 172.21.15.37:0/66380304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedfc15f380 msgr2=0x7fedfc0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.096+0000 7fee0091a640 1 --2- 172.21.15.37:0/66380304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedfc15f380 0x7fedfc0d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.096+0000 7fee0091a640 1 -- 172.21.15.37:0/66380304 >> v2:172.21.15.134:3300/0 conn(0x7fedfc0d5b50 msgr2=0x7fedfc0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.096+0000 7fee0091a640 1 --2- 172.21.15.37:0/66380304 >> v2:172.21.15.134:3300/0 conn(0x7fedfc0d5b50 0x7fedfc0cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.096+0000 7fee0091a640 1 -- 172.21.15.37:0/66380304 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc0b8280 con 0x7fedfc15f760 2024-04-26T20:26:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.096+0000 7fee0111b640 1 --2- 172.21.15.37:0/66380304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedfc15f380 0x7fedfc0d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:36.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.096+0000 7fee0091a640 1 --2- 172.21.15.37:0/66380304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedfc15f760 0x7fedfc0d2110 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fedfc08d560 tx=0x7fedf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb91656d7ef21dec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.096+0000 7fedf3fff640 1 -- 172.21.15.37:0/66380304 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8037070 con 0x7fedfc15f760 2024-04-26T20:26:36.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.096+0000 7fedf3fff640 1 -- 172.21.15.37:0/66380304 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf8032440 con 0x7fedfc15f760 2024-04-26T20:26:36.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.096+0000 7fedf3fff640 1 -- 172.21.15.37:0/66380304 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8032810 con 0x7fedfc15f760 2024-04-26T20:26:36.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.097+0000 7fee0337f640 1 -- 172.21.15.37:0/66380304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedfc15f760 msgr2=0x7fedfc0d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.097+0000 7fee0337f640 1 --2- 172.21.15.37:0/66380304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedfc15f760 0x7fedfc0d2110 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fedfc08d560 tx=0x7fedf802a270 comp rx=0 tx=0).stop 2024-04-26T20:26:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.097+0000 7fee0337f640 1 -- 172.21.15.37:0/66380304 shutdown_connections 2024-04-26T20:26:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.097+0000 7fee0337f640 1 --2- 172.21.15.37:0/66380304 >> v2:172.21.15.134:3300/0 conn(0x7fedfc0d5b50 0x7fedfc0cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.097+0000 7fee0337f640 1 --2- 172.21.15.37:0/66380304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedfc15f760 0x7fedfc0d2110 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.097+0000 7fee0337f640 1 --2- 172.21.15.37:0/66380304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedfc15f380 0x7fedfc0d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.097+0000 7fee0337f640 1 -- 172.21.15.37:0/66380304 >> 172.21.15.37:0/66380304 conn(0x7fedfc09e560 msgr2=0x7fedfc0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.097+0000 7fee0337f640 1 -- 172.21.15.37:0/66380304 shutdown_connections 2024-04-26T20:26:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.097+0000 7fee0337f640 1 -- 172.21.15.37:0/66380304 wait complete. 2024-04-26T20:26:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.098+0000 7fee0337f640 1 Processor -- start 2024-04-26T20:26:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.098+0000 7fee0337f640 1 -- start start 2024-04-26T20:26:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.099+0000 7fee0337f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedfc0d5b50 0x7fedfc1f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.099+0000 7fee0337f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedfc15f380 0x7fedfc0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.099+0000 7fee0337f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedfc15f760 0x7fedfc0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.099+0000 7fee0337f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc0d1280 con 0x7fedfc0d5b50 2024-04-26T20:26:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.099+0000 7fee0337f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc0d1100 con 0x7fedfc15f760 2024-04-26T20:26:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.099+0000 7fee0337f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fedfc0d3840 con 0x7fedfc15f380 2024-04-26T20:26:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.099+0000 7fee0111b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedfc0d5b50 0x7fedfc1f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.099+0000 7fee0111b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedfc0d5b50 0x7fedfc1f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41960/0 (socket says 172.21.15.37:41960) 2024-04-26T20:26:36.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.099+0000 7fee0111b640 1 -- 172.21.15.37:0/361754506 learned_addr learned my addr 172.21.15.37:0/361754506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:36.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.099+0000 7fee0191c640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedfc15f760 0x7fedfc0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.099+0000 7fee0091a640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedfc15f380 0x7fedfc0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.100+0000 7fee0111b640 1 -- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedfc15f380 msgr2=0x7fedfc0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.100+0000 7fee0111b640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedfc15f380 0x7fedfc0ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.100+0000 7fee0111b640 1 -- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedfc15f760 msgr2=0x7fedfc0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.100+0000 7fee0111b640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedfc15f760 0x7fedfc0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.100+0000 7fee0111b640 1 -- 172.21.15.37:0/361754506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc0cae90 con 0x7fedfc0d5b50 2024-04-26T20:26:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.100+0000 7fee0111b640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedfc0d5b50 0x7fedfc1f4490 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fedec00e7e0 tx=0x7fedec00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.100+0000 7fedf1ffb640 1 -- 172.21.15.37:0/361754506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec00f040 con 0x7fedfc0d5b50 2024-04-26T20:26:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.100+0000 7fedf1ffb640 1 -- 172.21.15.37:0/361754506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedec004030 con 0x7fedfc0d5b50 2024-04-26T20:26:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.100+0000 7fee0337f640 1 -- 172.21.15.37:0/361754506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc0cb170 con 0x7fedfc0d5b50 2024-04-26T20:26:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.100+0000 7fedf1ffb640 1 -- 172.21.15.37:0/361754506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec004400 con 0x7fedfc0d5b50 2024-04-26T20:26:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.100+0000 7fee0337f640 1 -- 172.21.15.37:0/361754506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc0cca20 con 0x7fedfc0d5b50 2024-04-26T20:26:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.101+0000 7fee0337f640 1 -- 172.21.15.37:0/361754506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedfc06a360 con 0x7fedfc0d5b50 2024-04-26T20:26:36.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.102+0000 7fedf1ffb640 1 -- 172.21.15.37:0/361754506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fedec00c6d0 con 0x7fedfc0d5b50 2024-04-26T20:26:36.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.102+0000 7fedf1ffb640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fede006cbc0 0x7fede006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.102+0000 7fedf1ffb640 1 -- 172.21.15.37:0/361754506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fedec08bad0 con 0x7fedfc0d5b50 2024-04-26T20:26:36.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.102+0000 7fee0091a640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fede006cbc0 0x7fede006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.103+0000 7fee0091a640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fede006cbc0 0x7fede006f080 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fedf8034000 tx=0x7fedf802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.106+0000 7fedf1ffb640 1 -- 172.21.15.37:0/361754506 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fedec017020 con 0x7fedfc0d5b50 2024-04-26T20:26:36.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.289+0000 7fee0337f640 1 -- 172.21.15.37:0/361754506 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fedfc076020 con 0x7fede006cbc0 2024-04-26T20:26:36.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.290+0000 7fedf1ffb640 1 -- 172.21.15.37:0/361754506 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fedfc076020 con 0x7fede006cbc0 2024-04-26T20:26:36.291 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T20:26:36.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 -- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fede006cbc0 msgr2=0x7fede006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fede006cbc0 0x7fede006f080 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fedf8034000 tx=0x7fedf802e000 comp rx=0 tx=0).stop 2024-04-26T20:26:36.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 -- 172.21.15.37:0/361754506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedfc0d5b50 msgr2=0x7fedfc1f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedfc0d5b50 0x7fedfc1f4490 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fedec00e7e0 tx=0x7fedec00eca0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 -- 172.21.15.37:0/361754506 shutdown_connections 2024-04-26T20:26:36.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedfc15f760 0x7fedfc0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fede006cbc0 0x7fede006f080 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedfc15f380 0x7fedfc0ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 --2- 172.21.15.37:0/361754506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedfc0d5b50 0x7fedfc1f4490 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 -- 172.21.15.37:0/361754506 >> 172.21.15.37:0/361754506 conn(0x7fedfc09e560 msgr2=0x7fedfc161be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 -- 172.21.15.37:0/361754506 shutdown_connections 2024-04-26T20:26:36.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.292+0000 7fee0337f640 1 -- 172.21.15.37:0/361754506 wait complete. 2024-04-26T20:26:36.303 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T20:26:36.304 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:26:36.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.406+0000 7f1eb735f640 1 Processor -- start 2024-04-26T20:26:36.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.407+0000 7f1eb735f640 1 -- start start 2024-04-26T20:26:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.407+0000 7f1eb735f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1eb015f410 0x7f1eb00b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.407+0000 7f1eb735f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eb015f7f0 0x7f1eb00d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.407+0000 7f1eb735f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb00b55a0 con 0x7f1eb00b50c0 2024-04-26T20:26:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.407+0000 7f1eb735f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb00b5880 con 0x7f1eb015f410 2024-04-26T20:26:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.407+0000 7f1eb735f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb00b5b60 con 0x7f1eb015f7f0 2024-04-26T20:26:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eb58fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1eb00b50c0 0x7f1eb00d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:35092/0 (socket says 172.21.15.37:35092) 2024-04-26T20:26:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eb58fc640 1 -- 172.21.15.37:0/1836253588 learned_addr learned my addr 172.21.15.37:0/1836253588 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eb48fa640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eb015f7f0 0x7f1eb00d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eb50fb640 1 --2- 172.21.15.37:0/1836253588 >> v2:172.21.15.134:3300/0 conn(0x7f1eb015f410 0x7f1eb00b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 157197792 0 0) 0x7f1eb00b55a0 con 0x7f1eb00b50c0 2024-04-26T20:26:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e98003520 con 0x7f1eb00b50c0 2024-04-26T20:26:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 129473910 0 0) 0x7f1e98003520 con 0x7f1eb00b50c0 2024-04-26T20:26:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eb015f7f0 msgr2=0x7f1eb00d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eb015f7f0 0x7f1eb00d3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 >> v2:172.21.15.134:3300/0 conn(0x7f1eb015f410 msgr2=0x7f1eb00b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 --2- 172.21.15.37:0/1836253588 >> v2:172.21.15.134:3300/0 conn(0x7f1eb015f410 0x7f1eb00b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb00b5e40 con 0x7f1eb00b50c0 2024-04-26T20:26:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f1ea00019f0 con 0x7f1eb00b50c0 2024-04-26T20:26:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 >> v1:172.21.15.37:6789/0 conn(0x7f1eb00b50c0 legacy=0x7f1eb00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e980039b0 0x7f1e98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e98004800 0x7f1e98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e980051c0 0x7f1e9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb00b5880 con 0x7f1e980039b0 2024-04-26T20:26:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb00b5b60 con 0x7f1e98004800 2024-04-26T20:26:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9800bf00 con 0x7f1e980051c0 2024-04-26T20:26:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eb50fb640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e980039b0 0x7f1e98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.408+0000 7f1eb58fc640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e980051c0 0x7f1e9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.409+0000 7f1eb48fa640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e98004800 0x7f1e98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.409+0000 7f1eb50fb640 1 -- 172.21.15.37:0/1836253588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e980051c0 msgr2=0x7f1e9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.409+0000 7f1eb50fb640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e980051c0 0x7f1e9800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.409+0000 7f1eb50fb640 1 -- 172.21.15.37:0/1836253588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e98004800 msgr2=0x7f1e98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.409+0000 7f1eb50fb640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e98004800 0x7f1e98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.409+0000 7f1eb50fb640 1 -- 172.21.15.37:0/1836253588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e98003520 con 0x7f1e980039b0 2024-04-26T20:26:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.409+0000 7f1eb50fb640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e980039b0 0x7f1e98003dd0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009ac0 tx=0x7f1e98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c238ca0133cc3f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.409+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8036070 con 0x7f1e980039b0 2024-04-26T20:26:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.410+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea8031400 con 0x7f1e980039b0 2024-04-26T20:26:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.410+0000 7f1eaffff640 1 -- 172.21.15.37:0/1836253588 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8009690 con 0x7f1e980039b0 2024-04-26T20:26:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.410+0000 7f1eb735f640 1 -- 172.21.15.37:0/1836253588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e980039b0 msgr2=0x7f1e98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.410+0000 7f1eb735f640 1 --2- 172.21.15.37:0/1836253588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e980039b0 0x7f1e98003dd0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009ac0 tx=0x7f1e98005830 comp rx=0 tx=0).stop 2024-04-26T20:26:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.410+0000 7f1eb58fc640 1 -- 172.21.15.37:0/1836253588 reap_dead start 2024-04-26T20:26:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.410+0000 7f1eb735f640 1 -- 172.21.15.37:0/1836253588 shutdown_connections 2024-04-26T20:26:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.410+0000 7f1eb735f640 1 -- 172.21.15.37:0/1836253588 >> 172.21.15.37:0/1836253588 conn(0x7f1eb008ebb0 msgr2=0x7f1eb009e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.410+0000 7f1eb735f640 1 -- 172.21.15.37:0/1836253588 shutdown_connections 2024-04-26T20:26:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.410+0000 7f1eb735f640 1 -- 172.21.15.37:0/1836253588 wait complete. 2024-04-26T20:26:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.411+0000 7f1eb735f640 1 Processor -- start 2024-04-26T20:26:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.411+0000 7f1eb735f640 1 -- start start 2024-04-26T20:26:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb735f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e980039b0 0x7f1eb00cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb735f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eb015f7f0 0x7f1eb015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb735f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eb0160190 0x7f1eb00cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb735f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb00d12a0 con 0x7f1eb015f7f0 2024-04-26T20:26:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb735f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb00d1120 con 0x7f1e980039b0 2024-04-26T20:26:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb735f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb00d1420 con 0x7f1eb0160190 2024-04-26T20:26:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb48fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eb015f7f0 0x7f1eb015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb48fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eb015f7f0 0x7f1eb015fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41992/0 (socket says 172.21.15.37:41992) 2024-04-26T20:26:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb48fa640 1 -- 172.21.15.37:0/1033616438 learned_addr learned my addr 172.21.15.37:0/1033616438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb50fb640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e980039b0 0x7f1eb00cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb58fc640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eb0160190 0x7f1eb00cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb48fa640 1 -- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eb0160190 msgr2=0x7f1eb00cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb48fa640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eb0160190 0x7f1eb00cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb48fa640 1 -- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e980039b0 msgr2=0x7f1eb00cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb48fa640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e980039b0 0x7f1eb00cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb48fa640 1 -- 172.21.15.37:0/1033616438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb0160610 con 0x7f1eb015f7f0 2024-04-26T20:26:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb50fb640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e980039b0 0x7f1eb00cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.412+0000 7f1eb58fc640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eb0160190 0x7f1eb00cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.413+0000 7f1eb48fa640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eb015f7f0 0x7f1eb015fc50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40119a0 tx=0x7f1ea4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.413+0000 7f1eadffb640 1 -- 172.21.15.37:0/1033616438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400faf0 con 0x7f1eb015f7f0 2024-04-26T20:26:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.413+0000 7f1eadffb640 1 -- 172.21.15.37:0/1033616438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea400fc90 con 0x7f1eb015f7f0 2024-04-26T20:26:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.413+0000 7f1eb735f640 1 -- 172.21.15.37:0/1033616438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb0160b00 con 0x7f1eb015f7f0 2024-04-26T20:26:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.413+0000 7f1eadffb640 1 -- 172.21.15.37:0/1033616438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4014440 con 0x7f1eb015f7f0 2024-04-26T20:26:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.413+0000 7f1eb735f640 1 -- 172.21.15.37:0/1033616438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb00c9c00 con 0x7f1eb015f7f0 2024-04-26T20:26:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.414+0000 7f1eb735f640 1 -- 172.21.15.37:0/1033616438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1eb006a360 con 0x7f1eb015f7f0 2024-04-26T20:26:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.414+0000 7f1eadffb640 1 -- 172.21.15.37:0/1033616438 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f1ea4012040 con 0x7f1eb015f7f0 2024-04-26T20:26:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.415+0000 7f1eadffb640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1e9c06cbc0 0x7f1e9c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.415+0000 7f1eadffb640 1 -- 172.21.15.37:0/1033616438 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f1ea408c510 con 0x7f1eb015f7f0 2024-04-26T20:26:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.415+0000 7f1eadffb640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] conn(0x7f1e9c072a80 0x7f1e9c074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.415+0000 7f1eadffb640 1 -- 172.21.15.37:0/1033616438 --> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ea4015e50 con 0x7f1e9c072a80 2024-04-26T20:26:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.415+0000 7f1eadffb640 1 -- 172.21.15.37:0/1033616438 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1ea4093050 con 0x7f1eb015f7f0 2024-04-26T20:26:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.415+0000 7f1eb50fb640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1e9c06cbc0 0x7f1e9c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.415+0000 7f1eb58fc640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] conn(0x7f1e9c072a80 0x7f1e9c074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.416+0000 7f1eb50fb640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1e9c06cbc0 0x7f1e9c06f080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8033000 tx=0x7f1ea8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.416+0000 7f1eb58fc640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] conn(0x7f1e9c072a80 0x7f1e9c074ea0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.417+0000 7f1eadffb640 1 -- 172.21.15.37:0/1033616438 <== osd.5 v2:172.21.15.134:6808/1335053617 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ea4015e50 con 0x7f1e9c072a80 2024-04-26T20:26:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.430+0000 7f1eb735f640 1 -- 172.21.15.37:0/1033616438 --> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1eb00ca050 con 0x7f1e9c072a80 2024-04-26T20:26:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eadffb640 1 -- 172.21.15.37:0/1033616438 <== osd.5 v2:172.21.15.134:6808/1335053617 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1eb00ca050 con 0x7f1e9c072a80 2024-04-26T20:26:36.432 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:26:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:26:36.432 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:26:36.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eb735f640 1 -- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] conn(0x7f1e9c072a80 msgr2=0x7f1e9c074ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eb735f640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:6808/1335053617,v1:172.21.15.134:6809/1335053617] conn(0x7f1e9c072a80 0x7f1e9c074ea0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eb735f640 1 -- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1e9c06cbc0 msgr2=0x7f1e9c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eb735f640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1e9c06cbc0 0x7f1e9c06f080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8033000 tx=0x7f1ea8008030 comp rx=0 tx=0).stop 2024-04-26T20:26:36.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eb735f640 1 -- 172.21.15.37:0/1033616438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eb015f7f0 msgr2=0x7f1eb015fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eb735f640 1 --2- 172.21.15.37:0/1033616438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1eb015f7f0 0x7f1eb015fc50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40119a0 tx=0x7f1ea4011e60 comp rx=0 tx=0).stop 2024-04-26T20:26:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eb58fc640 1 -- 172.21.15.37:0/1033616438 reap_dead start 2024-04-26T20:26:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eb735f640 1 -- 172.21.15.37:0/1033616438 shutdown_connections 2024-04-26T20:26:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eb735f640 1 -- 172.21.15.37:0/1033616438 >> 172.21.15.37:0/1033616438 conn(0x7f1eb008ebb0 msgr2=0x7f1eb00d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eb735f640 1 -- 172.21.15.37:0/1033616438 shutdown_connections 2024-04-26T20:26:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.431+0000 7f1eb735f640 1 -- 172.21.15.37:0/1033616438 wait complete. 2024-04-26T20:26:36.442 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T20:26:36.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.542+0000 7fdaf810a640 1 Processor -- start 2024-04-26T20:26:36.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.542+0000 7fdaf810a640 1 -- start start 2024-04-26T20:26:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf810a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdaf006f790 0x7fdaf0074030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf810a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b6b10 0x7fdaf0161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf810a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf0161b80 con 0x7fdaf0160d70 2024-04-26T20:26:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf810a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf00b4400 con 0x7fdaf006f790 2024-04-26T20:26:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf810a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf0074570 con 0x7fdaf00b6b10 2024-04-26T20:26:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf5ea6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdaf006f790 0x7fdaf0074030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf5ea6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdaf006f790 0x7fdaf0074030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42008/0 (socket says 172.21.15.37:42008) 2024-04-26T20:26:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf5ea6640 1 -- 172.21.15.37:0/2555312755 learned_addr learned my addr 172.21.15.37:0/2555312755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf66a7640 1 --2- 172.21.15.37:0/2555312755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b6b10 0x7fdaf0161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf5ea6640 1 -- 172.21.15.37:0/2555312755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b6b10 msgr2=0x7fdaf0161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf5ea6640 1 --2- 172.21.15.37:0/2555312755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b6b10 0x7fdaf0161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.543+0000 7fdaf5ea6640 1 -- 172.21.15.37:0/2555312755 >> v1:172.21.15.134:6789/0 conn(0x7fdaf0160d70 legacy=0x7fdaf015d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.544+0000 7fdaf5ea6640 1 -- 172.21.15.37:0/2555312755 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf00b4120 con 0x7fdaf006f790 2024-04-26T20:26:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.544+0000 7fdaf5ea6640 1 --2- 172.21.15.37:0/2555312755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf006f790 0x7fdaf0074030 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009870 tx=0x7fdae802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b75632b01e8c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.544+0000 7fdaf4ea4640 1 -- 172.21.15.37:0/2555312755 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8037070 con 0x7fdaf006f790 2024-04-26T20:26:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.544+0000 7fdaf4ea4640 1 -- 172.21.15.37:0/2555312755 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae8032440 con 0x7fdaf006f790 2024-04-26T20:26:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.544+0000 7fdaf4ea4640 1 -- 172.21.15.37:0/2555312755 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8032810 con 0x7fdaf006f790 2024-04-26T20:26:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.545+0000 7fdaf810a640 1 -- 172.21.15.37:0/2555312755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf006f790 msgr2=0x7fdaf0074030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.545+0000 7fdaf810a640 1 --2- 172.21.15.37:0/2555312755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf006f790 0x7fdaf0074030 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009870 tx=0x7fdae802a270 comp rx=0 tx=0).stop 2024-04-26T20:26:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.545+0000 7fdaf810a640 1 -- 172.21.15.37:0/2555312755 shutdown_connections 2024-04-26T20:26:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.545+0000 7fdaf810a640 1 --2- 172.21.15.37:0/2555312755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b6b10 0x7fdaf0161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.545+0000 7fdaf810a640 1 --2- 172.21.15.37:0/2555312755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf006f790 0x7fdaf0074030 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.545+0000 7fdaf810a640 1 -- 172.21.15.37:0/2555312755 >> 172.21.15.37:0/2555312755 conn(0x7fdaf00d0d50 msgr2=0x7fdaf00d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.545+0000 7fdaf810a640 1 -- 172.21.15.37:0/2555312755 shutdown_connections 2024-04-26T20:26:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.545+0000 7fdaf810a640 1 -- 172.21.15.37:0/2555312755 wait complete. 2024-04-26T20:26:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.546+0000 7fdaf810a640 1 Processor -- start 2024-04-26T20:26:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.546+0000 7fdaf810a640 1 -- start start 2024-04-26T20:26:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf810a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf006f790 0x7fdaf0146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf810a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf00b6b10 0x7fdaf00d0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf810a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf0160d70 0x7fdaf015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf810a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf00735b0 con 0x7fdaf00b6b10 2024-04-26T20:26:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf810a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf00732d0 con 0x7fdaf0160d70 2024-04-26T20:26:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf810a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf00b3d80 con 0x7fdaf006f790 2024-04-26T20:26:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf56a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf00b6b10 0x7fdaf00d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf56a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf00b6b10 0x7fdaf00d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42012/0 (socket says 172.21.15.37:42012) 2024-04-26T20:26:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf56a5640 1 -- 172.21.15.37:0/2232614978 learned_addr learned my addr 172.21.15.37:0/2232614978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf5ea6640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf006f790 0x7fdaf0146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf66a7640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf0160d70 0x7fdaf015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf56a5640 1 -- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf006f790 msgr2=0x7fdaf0146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf56a5640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf006f790 0x7fdaf0146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf56a5640 1 -- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf0160d70 msgr2=0x7fdaf015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf56a5640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf0160d70 0x7fdaf015bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.547+0000 7fdaf56a5640 1 -- 172.21.15.37:0/2232614978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf015ba70 con 0x7fdaf00b6b10 2024-04-26T20:26:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.548+0000 7fdaf56a5640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf00b6b10 0x7fdaf00d0210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fdae400c3c0 tx=0x7fdae400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.548+0000 7fdaee7fc640 1 -- 172.21.15.37:0/2232614978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae40052d0 con 0x7fdaf00b6b10 2024-04-26T20:26:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.548+0000 7fdaee7fc640 1 -- 172.21.15.37:0/2232614978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae4005490 con 0x7fdaf00b6b10 2024-04-26T20:26:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.548+0000 7fdaf810a640 1 -- 172.21.15.37:0/2232614978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf015b1d0 con 0x7fdaf00b6b10 2024-04-26T20:26:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.548+0000 7fdaee7fc640 1 -- 172.21.15.37:0/2232614978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4007260 con 0x7fdaf00b6b10 2024-04-26T20:26:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.548+0000 7fdaf810a640 1 -- 172.21.15.37:0/2232614978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf015b790 con 0x7fdaf00b6b10 2024-04-26T20:26:36.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.549+0000 7fdaf810a640 1 -- 172.21.15.37:0/2232614978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf006a360 con 0x7fdaf00b6b10 2024-04-26T20:26:36.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.549+0000 7fdaee7fc640 1 -- 172.21.15.37:0/2232614978 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fdae4007400 con 0x7fdaf00b6b10 2024-04-26T20:26:36.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.550+0000 7fdaee7fc640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdadc06cbc0 0x7fdadc06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.550+0000 7fdaee7fc640 1 -- 172.21.15.37:0/2232614978 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fdae408b150 con 0x7fdaf00b6b10 2024-04-26T20:26:36.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.550+0000 7fdaf5ea6640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdadc06cbc0 0x7fdadc06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.550+0000 7fdaf5ea6640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdadc06cbc0 0x7fdadc06f080 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdae8034000 tx=0x7fdae802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.554+0000 7fdaee7fc640 1 -- 172.21.15.37:0/2232614978 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdae40595c0 con 0x7fdaf00b6b10 2024-04-26T20:26:36.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.739+0000 7fdaf810a640 1 -- 172.21.15.37:0/2232614978 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fdaf006ff40 con 0x7fdadc06cbc0 2024-04-26T20:26:36.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.739+0000 7fdaee7fc640 1 -- 172.21.15.37:0/2232614978 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdaf006ff40 con 0x7fdadc06cbc0 2024-04-26T20:26:36.740 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T20:26:36.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 -- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdadc06cbc0 msgr2=0x7fdadc06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdadc06cbc0 0x7fdadc06f080 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdae8034000 tx=0x7fdae802e000 comp rx=0 tx=0).stop 2024-04-26T20:26:36.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 -- 172.21.15.37:0/2232614978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf00b6b10 msgr2=0x7fdaf00d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf00b6b10 0x7fdaf00d0210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fdae400c3c0 tx=0x7fdae400c880 comp rx=0 tx=0).stop 2024-04-26T20:26:36.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 -- 172.21.15.37:0/2232614978 shutdown_connections 2024-04-26T20:26:36.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf0160d70 0x7fdaf015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf00b6b10 0x7fdaf00d0210 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fdadc06cbc0 0x7fdadc06f080 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 --2- 172.21.15.37:0/2232614978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf006f790 0x7fdaf0146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 -- 172.21.15.37:0/2232614978 >> 172.21.15.37:0/2232614978 conn(0x7fdaf00d0d50 msgr2=0x7fdaf00d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 -- 172.21.15.37:0/2232614978 shutdown_connections 2024-04-26T20:26:36.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.741+0000 7fdaf810a640 1 -- 172.21.15.37:0/2232614978 wait complete. 2024-04-26T20:26:36.753 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T20:26:36.753 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:26:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.856+0000 7f2de4c52640 1 Processor -- start 2024-04-26T20:26:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.856+0000 7f2de4c52640 1 -- start start 2024-04-26T20:26:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2de4c52640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2de00cdf30 0x7f2de01638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2de4c52640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de00aedf0 con 0x7f2de0145910 2024-04-26T20:26:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2de4c52640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de00af0d0 con 0x7f2de00ae8c0 2024-04-26T20:26:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2de4c52640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2de006e880 con 0x7f2de00cdf30 2024-04-26T20:26:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dde59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2de0145910 0x7f2de00cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:35108/0 (socket says 172.21.15.37:35108) 2024-04-26T20:26:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dde59c640 1 -- 172.21.15.37:0/2217756916 learned_addr learned my addr 172.21.15.37:0/2217756916 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3789722512 0 0) 0x7f2de00aedf0 con 0x7f2de0145910 2024-04-26T20:26:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dcc003520 con 0x7f2de0145910 2024-04-26T20:26:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dd7fff640 1 --2- 172.21.15.37:0/2217756916 >> v2:172.21.15.134:3300/0 conn(0x7f2de00cdf30 0x7f2de01638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2191239674 0 0) 0x7f2dcc003520 con 0x7f2de0145910 2024-04-26T20:26:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 >> v2:172.21.15.134:3300/0 conn(0x7f2de00cdf30 msgr2=0x7f2de01638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 --2- 172.21.15.37:0/2217756916 >> v2:172.21.15.134:3300/0 conn(0x7f2de00cdf30 0x7f2de01638f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 >> v1:172.21.15.134:6789/0 conn(0x7f2de00ae8c0 legacy=0x7f2de00cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de00af3b0 con 0x7f2de0145910 2024-04-26T20:26:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 685430966 0 0) 0x7f2dd0001670 con 0x7f2de0145910 2024-04-26T20:26:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 >> v1:172.21.15.37:6789/0 conn(0x7f2de0145910 legacy=0x7f2de00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 --2- 172.21.15.37:0/2217756916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2dcc0039b0 0x7f2dcc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 --2- 172.21.15.37:0/2217756916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dcc004800 0x7f2dcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 --2- 172.21.15.37:0/2217756916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 0x7f2dcc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc00c0c0 con 0x7f2dcc0051c0 2024-04-26T20:26:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc00c270 con 0x7f2dcc004800 2024-04-26T20:26:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.857+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2dcc00c470 con 0x7f2dcc0039b0 2024-04-26T20:26:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.858+0000 7f2dded9d640 1 --2- 172.21.15.37:0/2217756916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 0x7f2dcc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.858+0000 7f2dde59c640 1 --2- 172.21.15.37:0/2217756916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2dcc0039b0 0x7f2dcc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.858+0000 7f2dd7fff640 1 --2- 172.21.15.37:0/2217756916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dcc004800 0x7f2dcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.858+0000 7f2dded9d640 1 -- 172.21.15.37:0/2217756916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2dcc0039b0 msgr2=0x7f2dcc003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.858+0000 7f2dded9d640 1 --2- 172.21.15.37:0/2217756916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2dcc0039b0 0x7f2dcc003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.858+0000 7f2dded9d640 1 -- 172.21.15.37:0/2217756916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dcc004800 msgr2=0x7f2dcc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.858+0000 7f2dded9d640 1 --2- 172.21.15.37:0/2217756916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dcc004800 0x7f2dcc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.858+0000 7f2dded9d640 1 -- 172.21.15.37:0/2217756916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dcc003520 con 0x7f2dcc0051c0 2024-04-26T20:26:36.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2dded9d640 1 -- 172.21.15.37:0/2217756916 reap_dead start 2024-04-26T20:26:36.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2dded9d640 1 --2- 172.21.15.37:0/2217756916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 0x7f2dcc00b9c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2de00d0de0 tx=0x7f2dcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a783a688ca945d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8004080 con 0x7f2dcc0051c0 2024-04-26T20:26:36.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd8006d70 con 0x7f2dcc0051c0 2024-04-26T20:26:36.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2dddd9b640 1 -- 172.21.15.37:0/2217756916 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd802ad70 con 0x7f2dcc0051c0 2024-04-26T20:26:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2de4c52640 1 -- 172.21.15.37:0/2217756916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 msgr2=0x7f2dcc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2de4c52640 1 --2- 172.21.15.37:0/2217756916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 0x7f2dcc00b9c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2de00d0de0 tx=0x7f2dcc005830 comp rx=0 tx=0).stop 2024-04-26T20:26:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2de4c52640 1 -- 172.21.15.37:0/2217756916 shutdown_connections 2024-04-26T20:26:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2de4c52640 1 --2- 172.21.15.37:0/2217756916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 0x7f2dcc00b9c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2de4c52640 1 -- 172.21.15.37:0/2217756916 >> 172.21.15.37:0/2217756916 conn(0x7f2de00b2c60 msgr2=0x7f2de00b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2de4c52640 1 -- 172.21.15.37:0/2217756916 shutdown_connections 2024-04-26T20:26:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.859+0000 7f2de4c52640 1 -- 172.21.15.37:0/2217756916 wait complete. 2024-04-26T20:26:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.861+0000 7f2de4c52640 1 Processor -- start 2024-04-26T20:26:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.861+0000 7f2de4c52640 1 -- start start 2024-04-26T20:26:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.861+0000 7f2de4c52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 0x7f2de0155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.861+0000 7f2de4c52640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2de00b0aa0 0x7f2de00b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.861+0000 7f2de4c52640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2de01f8b60 0x7f2de01f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.861+0000 7f2de4c52640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01638f0 con 0x7f2dcc0051c0 2024-04-26T20:26:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.861+0000 7f2de4c52640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2de0145cf0 con 0x7f2de01f8b60 2024-04-26T20:26:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.861+0000 7f2de4c52640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2de0163a70 con 0x7f2de00b0aa0 2024-04-26T20:26:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.861+0000 7f2dde59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 0x7f2de0155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dde59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 0x7f2de0155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42024/0 (socket says 172.21.15.37:42024) 2024-04-26T20:26:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dde59c640 1 -- 172.21.15.37:0/1043609910 learned_addr learned my addr 172.21.15.37:0/1043609910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dd7fff640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2de00b0aa0 0x7f2de00b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dded9d640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2de01f8b60 0x7f2de01f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dde59c640 1 -- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2de00b0aa0 msgr2=0x7f2de00b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dde59c640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2de00b0aa0 0x7f2de00b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dde59c640 1 -- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2de01f8b60 msgr2=0x7f2de01f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dde59c640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2de01f8b60 0x7f2de01f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dde59c640 1 -- 172.21.15.37:0/1043609910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de01f98d0 con 0x7f2dcc0051c0 2024-04-26T20:26:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dde59c640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 0x7f2de0155750 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0007340 tx=0x7f2dd0007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dd6ffd640 1 -- 172.21.15.37:0/1043609910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd001a070 con 0x7f2dcc0051c0 2024-04-26T20:26:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dd6ffd640 1 -- 172.21.15.37:0/1043609910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd000f860 con 0x7f2dcc0051c0 2024-04-26T20:26:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2de4c52640 1 -- 172.21.15.37:0/1043609910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de0072f80 con 0x7f2dcc0051c0 2024-04-26T20:26:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2dd6ffd640 1 -- 172.21.15.37:0/1043609910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000fc30 con 0x7f2dcc0051c0 2024-04-26T20:26:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.862+0000 7f2de4c52640 1 -- 172.21.15.37:0/1043609910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de0073540 con 0x7f2dcc0051c0 2024-04-26T20:26:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.863+0000 7f2de4c52640 1 -- 172.21.15.37:0/1043609910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2de0155c90 con 0x7f2dcc0051c0 2024-04-26T20:26:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.864+0000 7f2dd6ffd640 1 -- 172.21.15.37:0/1043609910 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f2dd001e3f0 con 0x7f2dcc0051c0 2024-04-26T20:26:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.864+0000 7f2dd6ffd640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2dc006cbf0 0x7f2dc006f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.864+0000 7f2dd6ffd640 1 -- 172.21.15.37:0/1043609910 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f2dd008b350 con 0x7f2dcc0051c0 2024-04-26T20:26:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.864+0000 7f2dd6ffd640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] conn(0x7f2dc0072ab0 0x7f2dc0074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.864+0000 7f2dd6ffd640 1 -- 172.21.15.37:0/1043609910 --> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2dc0075580 con 0x7f2dc0072ab0 2024-04-26T20:26:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.864+0000 7f2dd6ffd640 1 -- 172.21.15.37:0/1043609910 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2dd0092050 con 0x7f2dcc0051c0 2024-04-26T20:26:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.864+0000 7f2dd7fff640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2dc006cbf0 0x7f2dc006f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.864+0000 7f2dded9d640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] conn(0x7f2dc0072ab0 0x7f2dc0074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.865+0000 7f2dd7fff640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2dc006cbf0 0x7f2dc006f0b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2de00ccea0 tx=0x7f2dc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.865+0000 7f2dded9d640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] conn(0x7f2dc0072ab0 0x7f2dc0074ed0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.866+0000 7f2dd6ffd640 1 -- 172.21.15.37:0/1043609910 <== osd.6 v2:172.21.15.134:6816/1715563323 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2dc0075580 con 0x7f2dc0072ab0 2024-04-26T20:26:36.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.879+0000 7f2de4c52640 1 -- 172.21.15.37:0/1043609910 --> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2de00cd3a0 con 0x7f2dc0072ab0 2024-04-26T20:26:36.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.879+0000 7f2dd6ffd640 1 -- 172.21.15.37:0/1043609910 <== osd.6 v2:172.21.15.134:6816/1715563323 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2de00cd3a0 con 0x7f2dc0072ab0 2024-04-26T20:26:36.881 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:26:36.881 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:26:36.881 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:26:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.880+0000 7f2de4c52640 1 -- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] conn(0x7f2dc0072ab0 msgr2=0x7f2dc0074ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.880+0000 7f2de4c52640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:6816/1715563323,v1:172.21.15.134:6817/1715563323] conn(0x7f2dc0072ab0 0x7f2dc0074ed0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.880+0000 7f2de4c52640 1 -- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2dc006cbf0 msgr2=0x7f2dc006f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.880+0000 7f2de4c52640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f2dc006cbf0 0x7f2dc006f0b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2de00ccea0 tx=0x7f2dc8008040 comp rx=0 tx=0).stop 2024-04-26T20:26:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.880+0000 7f2de4c52640 1 -- 172.21.15.37:0/1043609910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 msgr2=0x7f2de0155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.880+0000 7f2de4c52640 1 --2- 172.21.15.37:0/1043609910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0051c0 0x7f2de0155750 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0007340 tx=0x7f2dd0007800 comp rx=0 tx=0).stop 2024-04-26T20:26:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.880+0000 7f2dded9d640 1 -- 172.21.15.37:0/1043609910 reap_dead start 2024-04-26T20:26:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.880+0000 7f2de4c52640 1 -- 172.21.15.37:0/1043609910 shutdown_connections 2024-04-26T20:26:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.880+0000 7f2de4c52640 1 -- 172.21.15.37:0/1043609910 >> 172.21.15.37:0/1043609910 conn(0x7f2de00b2c60 msgr2=0x7f2de00b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.880+0000 7f2de4c52640 1 -- 172.21.15.37:0/1043609910 shutdown_connections 2024-04-26T20:26:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.880+0000 7f2de4c52640 1 -- 172.21.15.37:0/1043609910 wait complete. 2024-04-26T20:26:36.891 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T20:26:36.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.991+0000 7f75bbb22640 1 Processor -- start 2024-04-26T20:26:36.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.992+0000 7f75bbb22640 1 -- start start 2024-04-26T20:26:36.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.992+0000 7f75bbb22640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b4162b40 0x7f75b40d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.992+0000 7f75bbb22640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f75b40d3c80 0x7f75b40d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.992+0000 7f75bbb22640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b40d0210 con 0x7f75b40d3820 2024-04-26T20:26:36.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.992+0000 7f75bbb22640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f75b40d04f0 con 0x7f75b40d3c80 2024-04-26T20:26:36.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.992+0000 7f75bbb22640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f75b40d07d0 con 0x7f75b4162b40 2024-04-26T20:26:36.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.992+0000 7f75ba0bf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f75b40d3c80 0x7f75b40d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.992+0000 7f75ba0bf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f75b40d3c80 0x7f75b40d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42038/0 (socket says 172.21.15.37:42038) 2024-04-26T20:26:36.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.993+0000 7f75ba0bf640 1 -- 172.21.15.37:0/1658073637 learned_addr learned my addr 172.21.15.37:0/1658073637 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:36.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.993+0000 7f75b98be640 1 --2- 172.21.15.37:0/1658073637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b4162b40 0x7f75b40d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.993+0000 7f75ba0bf640 1 -- 172.21.15.37:0/1658073637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b4162b40 msgr2=0x7f75b40d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.993+0000 7f75ba0bf640 1 --2- 172.21.15.37:0/1658073637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b4162b40 0x7f75b40d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.993+0000 7f75ba0bf640 1 -- 172.21.15.37:0/1658073637 >> v1:172.21.15.134:6789/0 conn(0x7f75b40d3820 legacy=0x7f75b40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.993+0000 7f75ba0bf640 1 -- 172.21.15.37:0/1658073637 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b40d0ab0 con 0x7f75b40d3c80 2024-04-26T20:26:36.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75ba0bf640 1 --2- 172.21.15.37:0/1658073637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75b40d3c80 0x7f75b40d1f60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009e50 tx=0x7f75a402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d560e2c844a45e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75b88bc640 1 -- 172.21.15.37:0/1658073637 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4036070 con 0x7f75b40d3c80 2024-04-26T20:26:36.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75b88bc640 1 -- 172.21.15.37:0/1658073637 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a4031600 con 0x7f75b40d3c80 2024-04-26T20:26:36.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75b88bc640 1 -- 172.21.15.37:0/1658073637 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a40319d0 con 0x7f75b40d3c80 2024-04-26T20:26:36.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75bbb22640 1 -- 172.21.15.37:0/1658073637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75b40d3c80 msgr2=0x7f75b40d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75bbb22640 1 --2- 172.21.15.37:0/1658073637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75b40d3c80 0x7f75b40d1f60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009e50 tx=0x7f75a402a7a0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75bbb22640 1 -- 172.21.15.37:0/1658073637 shutdown_connections 2024-04-26T20:26:36.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75bbb22640 1 --2- 172.21.15.37:0/1658073637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75b40d3c80 0x7f75b40d1f60 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75bbb22640 1 --2- 172.21.15.37:0/1658073637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b4162b40 0x7f75b40d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75bbb22640 1 -- 172.21.15.37:0/1658073637 >> 172.21.15.37:0/1658073637 conn(0x7f75b409e550 msgr2=0x7f75b40d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75bbb22640 1 -- 172.21.15.37:0/1658073637 shutdown_connections 2024-04-26T20:26:37.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.994+0000 7f75bbb22640 1 -- 172.21.15.37:0/1658073637 wait complete. 2024-04-26T20:26:37.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.995+0000 7f75bbb22640 1 Processor -- start 2024-04-26T20:26:37.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.996+0000 7f75bbb22640 1 -- start start 2024-04-26T20:26:37.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.996+0000 7f75bbb22640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b40d3820 0x7f75b41545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.996+0000 7f75bbb22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75b40d3c80 0x7f75b40cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.996+0000 7f75bbb22640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b4162b40 0x7f75b4155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.996+0000 7f75bbb22640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f75b40b5bb0 con 0x7f75b40d3c80 2024-04-26T20:26:37.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.996+0000 7f75bbb22640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75b40d4110 con 0x7f75b4162b40 2024-04-26T20:26:37.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.996+0000 7f75bbb22640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f75b40d6110 con 0x7f75b40d3820 2024-04-26T20:26:37.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.996+0000 7f75b90bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75b40d3c80 0x7f75b40cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.996+0000 7f75b90bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75b40d3c80 0x7f75b40cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42042/0 (socket says 172.21.15.37:42042) 2024-04-26T20:26:37.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.996+0000 7f75b90bd640 1 -- 172.21.15.37:0/375473857 learned_addr learned my addr 172.21.15.37:0/375473857 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:37.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75ba0bf640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b4162b40 0x7f75b4155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75b98be640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b40d3820 0x7f75b41545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75b90bd640 1 -- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b40d3820 msgr2=0x7f75b41545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75b90bd640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b40d3820 0x7f75b41545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75b90bd640 1 -- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b4162b40 msgr2=0x7f75b4155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75b90bd640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b4162b40 0x7f75b4155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75b90bd640 1 -- 172.21.15.37:0/375473857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b415fc00 con 0x7f75b40d3c80 2024-04-26T20:26:37.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75b90bd640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75b40d3c80 0x7f75b40cc790 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f75a800c3c0 tx=0x7f75a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75b27fc640 1 -- 172.21.15.37:0/375473857 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a80052d0 con 0x7f75b40d3c80 2024-04-26T20:26:37.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75b27fc640 1 -- 172.21.15.37:0/375473857 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a8005490 con 0x7f75b40d3c80 2024-04-26T20:26:37.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75bbb22640 1 -- 172.21.15.37:0/375473857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b415fee0 con 0x7f75b40d3c80 2024-04-26T20:26:37.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75b27fc640 1 -- 172.21.15.37:0/375473857 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8007260 con 0x7f75b40d3c80 2024-04-26T20:26:37.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.997+0000 7f75bbb22640 1 -- 172.21.15.37:0/375473857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b4154b20 con 0x7f75b40d3c80 2024-04-26T20:26:37.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.998+0000 7f75bbb22640 1 -- 172.21.15.37:0/375473857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b406a360 con 0x7f75b40d3c80 2024-04-26T20:26:37.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.998+0000 7f75b27fc640 1 -- 172.21.15.37:0/375473857 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f75a8007400 con 0x7f75b40d3c80 2024-04-26T20:26:37.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.999+0000 7f75b27fc640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f75a006cbc0 0x7f75a006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.999+0000 7f75b27fc640 1 -- 172.21.15.37:0/375473857 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f75a808b1b0 con 0x7f75b40d3c80 2024-04-26T20:26:37.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:36.999+0000 7f75b98be640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f75a006cbc0 0x7f75a006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.000+0000 7f75b98be640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f75a006cbc0 0x7f75a006f080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009a10 tx=0x7f75ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.003+0000 7f75b27fc640 1 -- 172.21.15.37:0/375473857 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75a8059540 con 0x7f75b40d3c80 2024-04-26T20:26:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.189+0000 7f75bbb22640 1 -- 172.21.15.37:0/375473857 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f75b406d8b0 con 0x7f75a006cbc0 2024-04-26T20:26:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.189+0000 7f75b27fc640 1 -- 172.21.15.37:0/375473857 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f75b406d8b0 con 0x7f75a006cbc0 2024-04-26T20:26:37.191 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T20:26:37.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 -- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f75a006cbc0 msgr2=0x7f75a006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f75a006cbc0 0x7f75a006f080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009a10 tx=0x7f75ac008040 comp rx=0 tx=0).stop 2024-04-26T20:26:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 -- 172.21.15.37:0/375473857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75b40d3c80 msgr2=0x7f75b40cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75b40d3c80 0x7f75b40cc790 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f75a800c3c0 tx=0x7f75a800c880 comp rx=0 tx=0).stop 2024-04-26T20:26:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 -- 172.21.15.37:0/375473857 shutdown_connections 2024-04-26T20:26:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b4162b40 0x7f75b4155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75b40d3c80 0x7f75b40cc790 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f75a006cbc0 0x7f75a006f080 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 --2- 172.21.15.37:0/375473857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b40d3820 0x7f75b41545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 -- 172.21.15.37:0/375473857 >> 172.21.15.37:0/375473857 conn(0x7f75b409e550 msgr2=0x7f75b40b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 -- 172.21.15.37:0/375473857 shutdown_connections 2024-04-26T20:26:37.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.192+0000 7f75bbb22640 1 -- 172.21.15.37:0/375473857 wait complete. 2024-04-26T20:26:37.204 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T20:26:37.204 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:26:37.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.307+0000 7fd2c6551640 1 Processor -- start 2024-04-26T20:26:37.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.307+0000 7fd2c6551640 1 -- start start 2024-04-26T20:26:37.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c6551640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2c015f410 0x7fd2c00b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c6551640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd2c015f7f0 0x7fd2c00d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c6551640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd2c00b50c0 0x7fd2c00d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c6551640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c00b55a0 con 0x7fd2c00b50c0 2024-04-26T20:26:37.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c6551640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c00b5880 con 0x7fd2c015f7f0 2024-04-26T20:26:37.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c6551640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c00b5b60 con 0x7fd2c015f410 2024-04-26T20:26:37.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c4aee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd2c00b50c0 0x7fd2c00d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c4aee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd2c00b50c0 0x7fd2c00d3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42050/0 (socket says 172.21.15.37:42050) 2024-04-26T20:26:37.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c4aee640 1 -- 172.21.15.37:0/33660646 learned_addr learned my addr 172.21.15.37:0/33660646 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:37.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2bffff640 1 --2- 172.21.15.37:0/33660646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2c015f410 0x7fd2c00b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2bf7fe640 1 --2- 172.21.15.37:0/33660646 >> v2:172.21.15.134:3300/0 conn(0x7fd2c015f7f0 0x7fd2c00d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c4aee640 1 -- 172.21.15.37:0/33660646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2c015f410 msgr2=0x7fd2c00b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:37.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c4aee640 1 --2- 172.21.15.37:0/33660646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2c015f410 0x7fd2c00b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c4aee640 1 -- 172.21.15.37:0/33660646 >> v2:172.21.15.134:3300/0 conn(0x7fd2c015f7f0 msgr2=0x7fd2c00d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:37.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c4aee640 1 --2- 172.21.15.37:0/33660646 >> v2:172.21.15.134:3300/0 conn(0x7fd2c015f7f0 0x7fd2c00d3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2c4aee640 1 -- 172.21.15.37:0/33660646 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c00b5e40 con 0x7fd2c00b50c0 2024-04-26T20:26:37.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2bffff640 1 --2- 172.21.15.37:0/33660646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2c015f410 0x7fd2c00b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:37.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.308+0000 7fd2bf7fe640 1 --2- 172.21.15.37:0/33660646 >> v2:172.21.15.134:3300/0 conn(0x7fd2c015f7f0 0x7fd2c00d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:37.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.309+0000 7fd2c4aee640 1 --2- 172.21.15.37:0/33660646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c00b50c0 0x7fd2c00d3cb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00d2d10 tx=0x7fd2b802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238b9d68b7e8a177 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.309+0000 7fd2beffd640 1 -- 172.21.15.37:0/33660646 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8036070 con 0x7fd2c00b50c0 2024-04-26T20:26:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.309+0000 7fd2beffd640 1 -- 172.21.15.37:0/33660646 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b8031600 con 0x7fd2c00b50c0 2024-04-26T20:26:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.309+0000 7fd2beffd640 1 -- 172.21.15.37:0/33660646 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80319d0 con 0x7fd2c00b50c0 2024-04-26T20:26:37.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.310+0000 7fd2c6551640 1 -- 172.21.15.37:0/33660646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c00b50c0 msgr2=0x7fd2c00d3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.310+0000 7fd2c6551640 1 --2- 172.21.15.37:0/33660646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c00b50c0 0x7fd2c00d3cb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00d2d10 tx=0x7fd2b802a730 comp rx=0 tx=0).stop 2024-04-26T20:26:37.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.310+0000 7fd2c6551640 1 -- 172.21.15.37:0/33660646 shutdown_connections 2024-04-26T20:26:37.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.310+0000 7fd2c6551640 1 --2- 172.21.15.37:0/33660646 >> v2:172.21.15.134:3300/0 conn(0x7fd2c015f7f0 0x7fd2c00d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.310+0000 7fd2c6551640 1 --2- 172.21.15.37:0/33660646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c00b50c0 0x7fd2c00d3cb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.310+0000 7fd2c6551640 1 --2- 172.21.15.37:0/33660646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2c015f410 0x7fd2c00b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.310+0000 7fd2c6551640 1 -- 172.21.15.37:0/33660646 >> 172.21.15.37:0/33660646 conn(0x7fd2c008ebb0 msgr2=0x7fd2c009e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.310+0000 7fd2c6551640 1 -- 172.21.15.37:0/33660646 shutdown_connections 2024-04-26T20:26:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.310+0000 7fd2c6551640 1 -- 172.21.15.37:0/33660646 wait complete. 2024-04-26T20:26:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.311+0000 7fd2c6551640 1 Processor -- start 2024-04-26T20:26:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.311+0000 7fd2c6551640 1 -- start start 2024-04-26T20:26:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2c6551640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c00b50c0 0x7fd2c0149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2c6551640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2c015f410 0x7fd2c00ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2c6551640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2c00cb600 0x7fd2c00ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2c6551640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c00d1cb0 con 0x7fd2c00b50c0 2024-04-26T20:26:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2c6551640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c00d1b30 con 0x7fd2c00cb600 2024-04-26T20:26:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2c6551640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c00d1e30 con 0x7fd2c015f410 2024-04-26T20:26:37.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c00b50c0 0x7fd2c0149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c00b50c0 0x7fd2c0149420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42054/0 (socket says 172.21.15.37:42054) 2024-04-26T20:26:37.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2bffff640 1 -- 172.21.15.37:0/1476713799 learned_addr learned my addr 172.21.15.37:0/1476713799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:37.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2bf7fe640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2c015f410 0x7fd2c00ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2bffff640 1 -- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2c015f410 msgr2=0x7fd2c00ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2bffff640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2c015f410 0x7fd2c00ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2c4aee640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2c00cb600 0x7fd2c00ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2bffff640 1 -- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2c00cb600 msgr2=0x7fd2c00ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.312+0000 7fd2bffff640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2c00cb600 0x7fd2c00ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.313+0000 7fd2bffff640 1 -- 172.21.15.37:0/1476713799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c01f5000 con 0x7fd2c00b50c0 2024-04-26T20:26:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.313+0000 7fd2bffff640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c00b50c0 0x7fd2c0149420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000e810 tx=0x7fd2b000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.313+0000 7fd2bcff9640 1 -- 172.21.15.37:0/1476713799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000f040 con 0x7fd2c00b50c0 2024-04-26T20:26:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.313+0000 7fd2bcff9640 1 -- 172.21.15.37:0/1476713799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b0004030 con 0x7fd2c00b50c0 2024-04-26T20:26:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.313+0000 7fd2c6551640 1 -- 172.21.15.37:0/1476713799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c00cbac0 con 0x7fd2c00b50c0 2024-04-26T20:26:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.313+0000 7fd2bcff9640 1 -- 172.21.15.37:0/1476713799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0004400 con 0x7fd2c00b50c0 2024-04-26T20:26:37.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.313+0000 7fd2c6551640 1 -- 172.21.15.37:0/1476713799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c00cdd60 con 0x7fd2c00b50c0 2024-04-26T20:26:37.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.314+0000 7fd2c6551640 1 -- 172.21.15.37:0/1476713799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2c006a360 con 0x7fd2c00b50c0 2024-04-26T20:26:37.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.314+0000 7fd2bcff9640 1 -- 172.21.15.37:0/1476713799 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fd2b000c6d0 con 0x7fd2c00b50c0 2024-04-26T20:26:37.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.315+0000 7fd2bcff9640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd2a006cbc0 0x7fd2a006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.315+0000 7fd2bcff9640 1 -- 172.21.15.37:0/1476713799 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fd2b008bba0 con 0x7fd2c00b50c0 2024-04-26T20:26:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.315+0000 7fd2bcff9640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] conn(0x7fd2a0072a80 0x7fd2a0074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.315+0000 7fd2bcff9640 1 -- 172.21.15.37:0/1476713799 --> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2a0075550 con 0x7fd2a0072a80 2024-04-26T20:26:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.315+0000 7fd2bcff9640 1 -- 172.21.15.37:0/1476713799 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd2b00902a0 con 0x7fd2c00b50c0 2024-04-26T20:26:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.315+0000 7fd2bf7fe640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd2a006cbc0 0x7fd2a006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.315+0000 7fd2c4aee640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] conn(0x7fd2a0072a80 0x7fd2a0074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.316+0000 7fd2bf7fe640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd2a006cbc0 0x7fd2a006f080 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd2c01492f0 tx=0x7fd2a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.316+0000 7fd2c4aee640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] conn(0x7fd2a0072a80 0x7fd2a0074ea0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.316+0000 7fd2bcff9640 1 -- 172.21.15.37:0/1476713799 <== osd.7 v2:172.21.15.134:6800/4008114040 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2a0075550 con 0x7fd2a0072a80 2024-04-26T20:26:37.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.329+0000 7fd2c6551640 1 -- 172.21.15.37:0/1476713799 --> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd2c00d2500 con 0x7fd2a0072a80 2024-04-26T20:26:37.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.330+0000 7fd2bcff9640 1 -- 172.21.15.37:0/1476713799 <== osd.7 v2:172.21.15.134:6800/4008114040 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd2c00d2500 con 0x7fd2a0072a80 2024-04-26T20:26:37.331 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T20:26:37.332 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:26:37.332 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T20:26:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.330+0000 7fd2c6551640 1 -- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] conn(0x7fd2a0072a80 msgr2=0x7fd2a0074ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.330+0000 7fd2c6551640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:6800/4008114040,v1:172.21.15.134:6801/4008114040] conn(0x7fd2a0072a80 0x7fd2a0074ea0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.330+0000 7fd2c6551640 1 -- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd2a006cbc0 msgr2=0x7fd2a006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.330+0000 7fd2c6551640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fd2a006cbc0 0x7fd2a006f080 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd2c01492f0 tx=0x7fd2a8008040 comp rx=0 tx=0).stop 2024-04-26T20:26:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.330+0000 7fd2c6551640 1 -- 172.21.15.37:0/1476713799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c00b50c0 msgr2=0x7fd2c0149420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.330+0000 7fd2c6551640 1 --2- 172.21.15.37:0/1476713799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c00b50c0 0x7fd2c0149420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000e810 tx=0x7fd2b000ecd0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.330+0000 7fd2c4aee640 1 -- 172.21.15.37:0/1476713799 reap_dead start 2024-04-26T20:26:37.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.330+0000 7fd2c6551640 1 -- 172.21.15.37:0/1476713799 shutdown_connections 2024-04-26T20:26:37.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.330+0000 7fd2c6551640 1 -- 172.21.15.37:0/1476713799 >> 172.21.15.37:0/1476713799 conn(0x7fd2c008ebb0 msgr2=0x7fd2c015e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.331+0000 7fd2c6551640 1 -- 172.21.15.37:0/1476713799 shutdown_connections 2024-04-26T20:26:37.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.331+0000 7fd2c6551640 1 -- 172.21.15.37:0/1476713799 wait complete. 2024-04-26T20:26:37.342 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T20:26:37.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.443+0000 7fa44d346640 1 Processor -- start 2024-04-26T20:26:37.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.443+0000 7fa44d346640 1 -- start start 2024-04-26T20:26:37.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa44d346640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4480d0540 0x7fa448072a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa44d346640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa448072f50 0x7fa44806ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa44d346640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44815be00 con 0x7fa44806f2c0 2024-04-26T20:26:37.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa44d346640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa44806f750 con 0x7fa448072f50 2024-04-26T20:26:37.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa44d346640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa44815c0a0 con 0x7fa4480d0540 2024-04-26T20:26:37.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa43ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa448072f50 0x7fa44806ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa43ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa448072f50 0x7fa44806ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42062/0 (socket says 172.21.15.37:42062) 2024-04-26T20:26:37.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa43ffff640 1 -- 172.21.15.37:0/3608709172 learned_addr learned my addr 172.21.15.37:0/3608709172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:37.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa446ffd640 1 --2- 172.21.15.37:0/3608709172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4480d0540 0x7fa448072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa43ffff640 1 -- 172.21.15.37:0/3608709172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4480d0540 msgr2=0x7fa448072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:37.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa43ffff640 1 --2- 172.21.15.37:0/3608709172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4480d0540 0x7fa448072a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa43ffff640 1 -- 172.21.15.37:0/3608709172 >> v1:172.21.15.134:6789/0 conn(0x7fa44806f2c0 legacy=0x7fa44815f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:37.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa43ffff640 1 -- 172.21.15.37:0/3608709172 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44815c380 con 0x7fa448072f50 2024-04-26T20:26:37.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.444+0000 7fa446ffd640 1 --2- 172.21.15.37:0/3608709172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4480d0540 0x7fa448072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:37.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.445+0000 7fa43ffff640 1 --2- 172.21.15.37:0/3608709172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa448072f50 0x7fa44806ed80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa4300098a0 tx=0x7fa43002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0285d43f3683813 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.445+0000 7fa4467fc640 1 -- 172.21.15.37:0/3608709172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430037070 con 0x7fa448072f50 2024-04-26T20:26:37.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.445+0000 7fa4467fc640 1 -- 172.21.15.37:0/3608709172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa430032440 con 0x7fa448072f50 2024-04-26T20:26:37.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.445+0000 7fa4467fc640 1 -- 172.21.15.37:0/3608709172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430032810 con 0x7fa448072f50 2024-04-26T20:26:37.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.446+0000 7fa44d346640 1 -- 172.21.15.37:0/3608709172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa448072f50 msgr2=0x7fa44806ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.446+0000 7fa44d346640 1 --2- 172.21.15.37:0/3608709172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa448072f50 0x7fa44806ed80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa4300098a0 tx=0x7fa43002a250 comp rx=0 tx=0).stop 2024-04-26T20:26:37.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.446+0000 7fa44d346640 1 -- 172.21.15.37:0/3608709172 shutdown_connections 2024-04-26T20:26:37.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.446+0000 7fa44d346640 1 --2- 172.21.15.37:0/3608709172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa448072f50 0x7fa44806ed80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.446+0000 7fa44d346640 1 --2- 172.21.15.37:0/3608709172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4480d0540 0x7fa448072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.446+0000 7fa44d346640 1 -- 172.21.15.37:0/3608709172 >> 172.21.15.37:0/3608709172 conn(0x7fa4480d0d50 msgr2=0x7fa4480d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.446+0000 7fa44d346640 1 -- 172.21.15.37:0/3608709172 shutdown_connections 2024-04-26T20:26:37.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.446+0000 7fa44d346640 1 -- 172.21.15.37:0/3608709172 wait complete. 2024-04-26T20:26:37.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.447+0000 7fa44d346640 1 Processor -- start 2024-04-26T20:26:37.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.447+0000 7fa44d346640 1 -- start start 2024-04-26T20:26:37.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa44d346640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa44806f2c0 0x7fa4481458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa44d346640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa448072f50 0x7fa44813a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa44d346640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4480d0540 0x7fa44814f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa44d346640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4480d0920 con 0x7fa4480d0540 2024-04-26T20:26:37.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa44d346640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa44815c940 con 0x7fa448072f50 2024-04-26T20:26:37.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa44d346640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa4480d0aa0 con 0x7fa44806f2c0 2024-04-26T20:26:37.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa4477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4480d0540 0x7fa44814f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa4477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4480d0540 0x7fa44814f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42076/0 (socket says 172.21.15.37:42076) 2024-04-26T20:26:37.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa4477fe640 1 -- 172.21.15.37:0/131379537 learned_addr learned my addr 172.21.15.37:0/131379537 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:37.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa43ffff640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa448072f50 0x7fa44813a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa446ffd640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa44806f2c0 0x7fa4481458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa4477fe640 1 -- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa44806f2c0 msgr2=0x7fa4481458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa4477fe640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa44806f2c0 0x7fa4481458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa4477fe640 1 -- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa448072f50 msgr2=0x7fa44813a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa4477fe640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa448072f50 0x7fa44813a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa4477fe640 1 -- 172.21.15.37:0/131379537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44813ac50 con 0x7fa4480d0540 2024-04-26T20:26:37.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa446ffd640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa44806f2c0 0x7fa4481458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:37.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa4477fe640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4480d0540 0x7fa44814f280 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c7b0 tx=0x7fa44000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.448+0000 7fa43f7fe640 1 -- 172.21.15.37:0/131379537 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440004b60 con 0x7fa4480d0540 2024-04-26T20:26:37.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.449+0000 7fa43f7fe640 1 -- 172.21.15.37:0/131379537 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa440004d00 con 0x7fa4480d0540 2024-04-26T20:26:37.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.449+0000 7fa44d346640 1 -- 172.21.15.37:0/131379537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4480b4f30 con 0x7fa4480d0540 2024-04-26T20:26:37.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.449+0000 7fa43f7fe640 1 -- 172.21.15.37:0/131379537 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44000aa30 con 0x7fa4480d0540 2024-04-26T20:26:37.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.449+0000 7fa44d346640 1 -- 172.21.15.37:0/131379537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4480b54f0 con 0x7fa4480d0540 2024-04-26T20:26:37.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.449+0000 7fa44d346640 1 -- 172.21.15.37:0/131379537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44806a360 con 0x7fa4480d0540 2024-04-26T20:26:37.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.450+0000 7fa43f7fe640 1 -- 172.21.15.37:0/131379537 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7fa44000abd0 con 0x7fa4480d0540 2024-04-26T20:26:37.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.450+0000 7fa43f7fe640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa42806cbc0 0x7fa42806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.450+0000 7fa43f7fe640 1 -- 172.21.15.37:0/131379537 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fa44008bee0 con 0x7fa4480d0540 2024-04-26T20:26:37.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.451+0000 7fa446ffd640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa42806cbc0 0x7fa42806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.451+0000 7fa446ffd640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa42806cbc0 0x7fa42806f080 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa438006fd0 tx=0x7fa438008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.455+0000 7fa43f7fe640 1 -- 172.21.15.37:0/131379537 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa440020070 con 0x7fa4480d0540 2024-04-26T20:26:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.639+0000 7fa44d346640 1 -- 172.21.15.37:0/131379537 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fa44806f8e0 con 0x7fa42806cbc0 2024-04-26T20:26:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.639+0000 7fa43f7fe640 1 -- 172.21.15.37:0/131379537 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa44806f8e0 con 0x7fa42806cbc0 2024-04-26T20:26:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T20:26:37.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 -- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa42806cbc0 msgr2=0x7fa42806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa42806cbc0 0x7fa42806f080 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa438006fd0 tx=0x7fa438008040 comp rx=0 tx=0).stop 2024-04-26T20:26:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 -- 172.21.15.37:0/131379537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4480d0540 msgr2=0x7fa44814f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4480d0540 0x7fa44814f280 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c7b0 tx=0x7fa44000cc70 comp rx=0 tx=0).stop 2024-04-26T20:26:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 -- 172.21.15.37:0/131379537 shutdown_connections 2024-04-26T20:26:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4480d0540 0x7fa44814f280 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa448072f50 0x7fa44813a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7fa42806cbc0 0x7fa42806f080 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 --2- 172.21.15.37:0/131379537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa44806f2c0 0x7fa4481458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 -- 172.21.15.37:0/131379537 >> 172.21.15.37:0/131379537 conn(0x7fa4480d0d50 msgr2=0x7fa44815e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 -- 172.21.15.37:0/131379537 shutdown_connections 2024-04-26T20:26:37.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.641+0000 7fa44d346640 1 -- 172.21.15.37:0/131379537 wait complete. 2024-04-26T20:26:37.653 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:26:37.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.753+0000 7f6749c07640 1 Processor -- start 2024-04-26T20:26:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.753+0000 7f6749c07640 1 -- start start 2024-04-26T20:26:37.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.754+0000 7f6749c07640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6744162b40 0x7f67440d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.754+0000 7f6749c07640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67440d2670 con 0x7f67440d3c80 2024-04-26T20:26:37.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.754+0000 7f6749c07640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67440d0210 con 0x7f67440d3820 2024-04-26T20:26:37.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.754+0000 7f6749c07640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f67440d04f0 con 0x7f6744162b40 2024-04-26T20:26:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.754+0000 7f6743fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f67440d3c80 0x7f67440d1f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:35112/0 (socket says 172.21.15.37:35112) 2024-04-26T20:26:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.754+0000 7f6743fff640 1 -- 172.21.15.37:0/1184248497 learned_addr learned my addr 172.21.15.37:0/1184248497 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.755+0000 7f67437fe640 1 --2- 172.21.15.37:0/1184248497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6744162b40 0x7f67440d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.755+0000 7f67427fc640 1 -- 172.21.15.37:0/1184248497 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3535585795 0 0) 0x7f67440d2670 con 0x7f67440d3c80 2024-04-26T20:26:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.755+0000 7f67427fc640 1 -- 172.21.15.37:0/1184248497 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f672c003520 con 0x7f67440d3c80 2024-04-26T20:26:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.755+0000 7f67427fc640 1 -- 172.21.15.37:0/1184248497 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 492776002 0 0) 0x7f67440d0210 con 0x7f67440d3820 2024-04-26T20:26:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.755+0000 7f67427fc640 1 -- 172.21.15.37:0/1184248497 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67440d2670 con 0x7f67440d3820 2024-04-26T20:26:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.755+0000 7f67427fc640 1 -- 172.21.15.37:0/1184248497 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 501589092 0 0) 0x7f672c003520 con 0x7f67440d3c80 2024-04-26T20:26:37.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.755+0000 7f67437fe640 1 -- 172.21.15.37:0/1184248497 >> v1:172.21.15.134:6789/0 conn(0x7f67440d3820 legacy=0x7f67440d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.755+0000 7f67437fe640 1 -- 172.21.15.37:0/1184248497 >> v1:172.21.15.37:6789/0 conn(0x7f67440d3c80 legacy=0x7f67440d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.755+0000 7f67437fe640 1 -- 172.21.15.37:0/1184248497 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67440d07d0 con 0x7f6744162b40 2024-04-26T20:26:37.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.756+0000 7f67437fe640 1 --2- 172.21.15.37:0/1184248497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6744162b40 0x7f67440d5bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f67340097d0 tx=0x7f673402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ade78cadc2091f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.756+0000 7f67427fc640 1 -- 172.21.15.37:0/1184248497 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734037070 con 0x7f6744162b40 2024-04-26T20:26:37.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.756+0000 7f67427fc640 1 -- 172.21.15.37:0/1184248497 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673402acc0 con 0x7f6744162b40 2024-04-26T20:26:37.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.756+0000 7f67427fc640 1 -- 172.21.15.37:0/1184248497 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c0019f0 con 0x7f6744162b40 2024-04-26T20:26:37.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.757+0000 7f6749c07640 1 -- 172.21.15.37:0/1184248497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6744162b40 msgr2=0x7f67440d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.757+0000 7f6749c07640 1 --2- 172.21.15.37:0/1184248497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6744162b40 0x7f67440d5bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f67340097d0 tx=0x7f673402a230 comp rx=0 tx=0).stop 2024-04-26T20:26:37.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.757+0000 7f6749c07640 1 -- 172.21.15.37:0/1184248497 shutdown_connections 2024-04-26T20:26:37.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.757+0000 7f6749c07640 1 --2- 172.21.15.37:0/1184248497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6744162b40 0x7f67440d5bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.757+0000 7f6749c07640 1 -- 172.21.15.37:0/1184248497 >> 172.21.15.37:0/1184248497 conn(0x7f674409e550 msgr2=0x7f67440d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.757+0000 7f6749c07640 1 -- 172.21.15.37:0/1184248497 shutdown_connections 2024-04-26T20:26:37.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.757+0000 7f6749c07640 1 -- 172.21.15.37:0/1184248497 wait complete. 2024-04-26T20:26:37.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.758+0000 7f6749c07640 1 Processor -- start 2024-04-26T20:26:37.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.758+0000 7f6749c07640 1 -- start start 2024-04-26T20:26:37.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f6749c07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67440d3820 0x7f67440ba780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f6749c07640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67440d3c80 0x7f67440cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f6749c07640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6744162b40 0x7f674415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f6749c07640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f67440d4110 con 0x7f67440d3820 2024-04-26T20:26:37.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f6749c07640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67440d0d90 con 0x7f6744162b40 2024-04-26T20:26:37.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f6749c07640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f67440b5bb0 con 0x7f67440d3c80 2024-04-26T20:26:37.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f67437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67440d3820 0x7f67440ba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f67437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67440d3820 0x7f67440ba780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42078/0 (socket says 172.21.15.37:42078) 2024-04-26T20:26:37.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f67437fe640 1 -- 172.21.15.37:0/669584655 learned_addr learned my addr 172.21.15.37:0/669584655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:37.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f6743fff640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6744162b40 0x7f674415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f6742ffd640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67440d3c80 0x7f67440cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f67437fe640 1 -- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67440d3c80 msgr2=0x7f67440cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f67437fe640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67440d3c80 0x7f67440cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f67437fe640 1 -- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6744162b40 msgr2=0x7f674415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f67437fe640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6744162b40 0x7f674415a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f67437fe640 1 -- 172.21.15.37:0/669584655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67440bb060 con 0x7f67440d3820 2024-04-26T20:26:37.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f6743fff640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6744162b40 0x7f674415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:37.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f67437fe640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67440d3820 0x7f67440ba780 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6734035040 tx=0x7f67340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.759+0000 7f672bfff640 1 -- 172.21.15.37:0/669584655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734037050 con 0x7f67440d3820 2024-04-26T20:26:37.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.760+0000 7f672bfff640 1 -- 172.21.15.37:0/669584655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6734043070 con 0x7f67440d3820 2024-04-26T20:26:37.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.760+0000 7f6749c07640 1 -- 172.21.15.37:0/669584655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67440cf0b0 con 0x7f67440d3820 2024-04-26T20:26:37.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.760+0000 7f672bfff640 1 -- 172.21.15.37:0/669584655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734006e10 con 0x7f67440d3820 2024-04-26T20:26:37.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.760+0000 7f6749c07640 1 -- 172.21.15.37:0/669584655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67440c9660 con 0x7f67440d3820 2024-04-26T20:26:37.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.760+0000 7f6749c07640 1 -- 172.21.15.37:0/669584655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674406a360 con 0x7f67440d3820 2024-04-26T20:26:37.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.761+0000 7f672bfff640 1 -- 172.21.15.37:0/669584655 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f673403ca30 con 0x7f67440d3820 2024-04-26T20:26:37.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.761+0000 7f672bfff640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f673006cbc0 0x7f673006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.761+0000 7f672bfff640 1 -- 172.21.15.37:0/669584655 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f67340ac700 con 0x7f67440d3820 2024-04-26T20:26:37.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.762+0000 7f6742ffd640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f673006cbc0 0x7f673006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.762+0000 7f6742ffd640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f673006cbc0 0x7f673006f080 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f67440cf310 tx=0x7f6738009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.766+0000 7f672bfff640 1 -- 172.21.15.37:0/669584655 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6734030e20 con 0x7f67440d3820 2024-04-26T20:26:37.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.946+0000 7f6749c07640 1 -- 172.21.15.37:0/669584655 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67440cf750 con 0x7f673006cbc0 2024-04-26T20:26:37.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.947+0000 7f672bfff640 1 -- 172.21.15.37:0/669584655 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13352 (secure 0 0 0) 0x7f67440cf750 con 0x7f673006cbc0 2024-04-26T20:26:37.948 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:37.949 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:26:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 -- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f673006cbc0 msgr2=0x7f673006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f673006cbc0 0x7f673006f080 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f67440cf310 tx=0x7f6738009040 comp rx=0 tx=0).stop 2024-04-26T20:26:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 -- 172.21.15.37:0/669584655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67440d3820 msgr2=0x7f67440ba780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67440d3820 0x7f67440ba780 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6734035040 tx=0x7f67340046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 -- 172.21.15.37:0/669584655 shutdown_connections 2024-04-26T20:26:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6744162b40 0x7f674415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f673006cbc0 0x7f673006f080 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67440d3c80 0x7f67440cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 --2- 172.21.15.37:0/669584655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67440d3820 0x7f67440ba780 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 -- 172.21.15.37:0/669584655 >> 172.21.15.37:0/669584655 conn(0x7f674409e550 msgr2=0x7f67440b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 -- 172.21.15.37:0/669584655 shutdown_connections 2024-04-26T20:26:37.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:37.949+0000 7f6749c07640 1 -- 172.21.15.37:0/669584655 wait complete. 2024-04-26T20:26:37.962 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-04-26T20:26:37.404959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238740,"kb_used_data":5928,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735980,"statfs":{"total":773094113280,"available":772849643520,"internally_reserved":0,"allocated":6070272,"data_stored":3196448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.723431"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":230,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T20:26:30.857415+0000","last_change":"2024-04-26T20:21:38.423615+0000","last_active":"2024-04-26T20:26:30.857415+0000","last_peered":"2024-04-26T20:26:30.857415+0000","last_clean":"2024-04-26T20:26:30.857415+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:26:30.857415+0000","last_undegraded":"2024-04-26T20:26:30.857415+0000","last_fullsized":"2024-04-26T20:26:30.857415+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:21:37.406582+0000","last_clean_scrub_stamp":"2024-04-26T20:21:37.406582+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:04.380524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1204224,"data_stored":842332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":842332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:26:37.964 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T20:21:37.406582+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=20, tm_min=21, tm_sec=37, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=20, tm_min=26, tm_sec=33, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T20:26:37.964 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T20:26:50.413 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:26:50.417 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:26:57.965 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:26:58.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.077+0000 7f36c8e7b640 1 Processor -- start 2024-04-26T20:26:58.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.077+0000 7f36c8e7b640 1 -- start start 2024-04-26T20:26:58.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36c8e7b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36c4142110 0x7f36c40ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36c8e7b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f36c40cf070 0x7f36c40aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36c8e7b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36c40af7d0 con 0x7f36c40aefc0 2024-04-26T20:26:58.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36c8e7b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f36c40cf4b0 con 0x7f36c40cf070 2024-04-26T20:26:58.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36c8e7b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f36c40afab0 con 0x7f36c4142110 2024-04-26T20:26:58.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36bbfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f36c40cf070 0x7f36c40aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36c2d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f36c40aefc0 0x7f36c4144940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45718/0 (socket says 172.21.15.37:45718) 2024-04-26T20:26:58.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36c2d9d640 1 -- 172.21.15.37:0/2097331244 learned_addr learned my addr 172.21.15.37:0/2097331244 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:58.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36c259c640 1 --2- 172.21.15.37:0/2097331244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36c4142110 0x7f36c40ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36bbfff640 1 -- 172.21.15.37:0/2097331244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36c4142110 msgr2=0x7f36c40ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:58.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36bbfff640 1 --2- 172.21.15.37:0/2097331244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36c4142110 0x7f36c40ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36bbfff640 1 -- 172.21.15.37:0/2097331244 >> v1:172.21.15.37:6789/0 conn(0x7f36c40aefc0 legacy=0x7f36c4144940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36bbfff640 1 -- 172.21.15.37:0/2097331244 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c40afd90 con 0x7f36c40cf070 2024-04-26T20:26:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36c1d9b640 1 -- 172.21.15.37:0/2097331244 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1359217732 0 0) 0x7f36c40af7d0 con 0x7f36c40aefc0 2024-04-26T20:26:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.078+0000 7f36c259c640 1 --2- 172.21.15.37:0/2097331244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36c4142110 0x7f36c40ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.079+0000 7f36bbfff640 1 --2- 172.21.15.37:0/2097331244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40cf070 0x7f36c40aea80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0097a0 tx=0x7f36ac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209bfb303ea0d09f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.079+0000 7f36c1d9b640 1 -- 172.21.15.37:0/2097331244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac037070 con 0x7f36c40cf070 2024-04-26T20:26:58.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.079+0000 7f36c1d9b640 1 -- 172.21.15.37:0/2097331244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36ac032440 con 0x7f36c40cf070 2024-04-26T20:26:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.079+0000 7f36c1d9b640 1 -- 172.21.15.37:0/2097331244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac032810 con 0x7f36c40cf070 2024-04-26T20:26:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.080+0000 7f36c8e7b640 1 -- 172.21.15.37:0/2097331244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40cf070 msgr2=0x7f36c40aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.080+0000 7f36c8e7b640 1 --2- 172.21.15.37:0/2097331244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40cf070 0x7f36c40aea80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0097a0 tx=0x7f36ac02a210 comp rx=0 tx=0).stop 2024-04-26T20:26:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.080+0000 7f36c8e7b640 1 -- 172.21.15.37:0/2097331244 shutdown_connections 2024-04-26T20:26:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.080+0000 7f36c8e7b640 1 --2- 172.21.15.37:0/2097331244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40cf070 0x7f36c40aea80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.080+0000 7f36c8e7b640 1 --2- 172.21.15.37:0/2097331244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36c4142110 0x7f36c40ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.080+0000 7f36c8e7b640 1 -- 172.21.15.37:0/2097331244 >> 172.21.15.37:0/2097331244 conn(0x7f36c40d0d50 msgr2=0x7f36c40d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.080+0000 7f36c8e7b640 1 -- 172.21.15.37:0/2097331244 shutdown_connections 2024-04-26T20:26:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.080+0000 7f36c8e7b640 1 -- 172.21.15.37:0/2097331244 wait complete. 2024-04-26T20:26:58.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.081+0000 7f36c8e7b640 1 Processor -- start 2024-04-26T20:26:58.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.081+0000 7f36c8e7b640 1 -- start start 2024-04-26T20:26:58.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.082+0000 7f36c8e7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40aefc0 0x7f36c4146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.082+0000 7f36c8e7b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36c40cf070 0x7f36c413e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.082+0000 7f36c8e7b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c4142110 0x7f36c4158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.082+0000 7f36c8e7b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36c40af430 con 0x7f36c40aefc0 2024-04-26T20:26:58.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.082+0000 7f36c8e7b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41424f0 con 0x7f36c4142110 2024-04-26T20:26:58.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.082+0000 7f36c8e7b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f36c40af5b0 con 0x7f36c40cf070 2024-04-26T20:26:58.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.082+0000 7f36c259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40aefc0 0x7f36c4146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.082+0000 7f36c259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40aefc0 0x7f36c4146a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38814/0 (socket says 172.21.15.37:38814) 2024-04-26T20:26:58.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.082+0000 7f36c259c640 1 -- 172.21.15.37:0/3733558677 learned_addr learned my addr 172.21.15.37:0/3733558677 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:58.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.082+0000 7f36bbfff640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36c40cf070 0x7f36c413e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.082+0000 7f36c2d9d640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c4142110 0x7f36c4158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.083+0000 7f36c259c640 1 -- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36c40cf070 msgr2=0x7f36c413e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.083+0000 7f36c259c640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36c40cf070 0x7f36c413e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.083+0000 7f36c259c640 1 -- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c4142110 msgr2=0x7f36c4158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.083+0000 7f36c259c640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c4142110 0x7f36c4158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.083+0000 7f36c259c640 1 -- 172.21.15.37:0/3733558677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c413baa0 con 0x7f36c40aefc0 2024-04-26T20:26:58.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.083+0000 7f36c259c640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40aefc0 0x7f36c4146a10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c7f0 tx=0x7f36b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.083+0000 7f36baffd640 1 -- 172.21.15.37:0/3733558677 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4015070 con 0x7f36c40aefc0 2024-04-26T20:26:58.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.083+0000 7f36baffd640 1 -- 172.21.15.37:0/3733558677 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b4007820 con 0x7f36c40aefc0 2024-04-26T20:26:58.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.083+0000 7f36baffd640 1 -- 172.21.15.37:0/3733558677 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4007bd0 con 0x7f36c40aefc0 2024-04-26T20:26:58.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.083+0000 7f36c8e7b640 1 -- 172.21.15.37:0/3733558677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c4147110 con 0x7f36c40aefc0 2024-04-26T20:26:58.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.083+0000 7f36c8e7b640 1 -- 172.21.15.37:0/3733558677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c41476d0 con 0x7f36c40aefc0 2024-04-26T20:26:58.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.084+0000 7f36c8e7b640 1 -- 172.21.15.37:0/3733558677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c406a360 con 0x7f36c40aefc0 2024-04-26T20:26:58.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.085+0000 7f36baffd640 1 -- 172.21.15.37:0/3733558677 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f36b4018070 con 0x7f36c40aefc0 2024-04-26T20:26:58.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.085+0000 7f36baffd640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f36a406cbc0 0x7f36a406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.085+0000 7f36baffd640 1 -- 172.21.15.37:0/3733558677 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f36b408bea0 con 0x7f36c40aefc0 2024-04-26T20:26:58.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.085+0000 7f36bbfff640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f36a406cbc0 0x7f36a406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.086+0000 7f36bbfff640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f36a406cbc0 0x7f36a406f080 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f36ac034000 tx=0x7f36ac02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.090+0000 7f36baffd640 1 -- 172.21.15.37:0/3733558677 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36b4059a90 con 0x7f36c40aefc0 2024-04-26T20:26:58.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.271+0000 7f36c8e7b640 1 -- 172.21.15.37:0/3733558677 --> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36c40b2020 con 0x7f36a406cbc0 2024-04-26T20:26:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.272+0000 7f36baffd640 1 -- 172.21.15.37:0/3733558677 <== mgr.4100 v2:172.21.15.134:6832/2709823969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7f36c40b2020 con 0x7f36a406cbc0 2024-04-26T20:26:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:58.274 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T20:26:58.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 -- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f36a406cbc0 msgr2=0x7f36a406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f36a406cbc0 0x7f36a406f080 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f36ac034000 tx=0x7f36ac02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:58.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 -- 172.21.15.37:0/3733558677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40aefc0 msgr2=0x7f36c4146a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40aefc0 0x7f36c4146a10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c7f0 tx=0x7f36b400ccb0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 -- 172.21.15.37:0/3733558677 shutdown_connections 2024-04-26T20:26:58.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c4142110 0x7f36c4158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f36a406cbc0 0x7f36a406f080 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36c40cf070 0x7f36c413e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 --2- 172.21.15.37:0/3733558677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40aefc0 0x7f36c4146a10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 -- 172.21.15.37:0/3733558677 >> 172.21.15.37:0/3733558677 conn(0x7f36c40d0d50 msgr2=0x7f36c406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:58.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 -- 172.21.15.37:0/3733558677 shutdown_connections 2024-04-26T20:26:58.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.274+0000 7f36c8e7b640 1 -- 172.21.15.37:0/3733558677 wait complete. 2024-04-26T20:26:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-04-26T20:26:57.409518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238720,"kb_used_data":5928,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736000,"statfs":{"total":773094113280,"available":772849664000,"internally_reserved":0,"allocated":6070272,"data_stored":3196448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002771"},"pg_stats":[{"pgid":"1.0","version":"51'119","reported_seq":241,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T20:26:37.964864+0000","last_change":"2024-04-26T20:26:37.964902+0000","last_active":"2024-04-26T20:26:37.964864+0000","last_peered":"2024-04-26T20:26:37.964864+0000","last_clean":"2024-04-26T20:26:37.964864+0000","last_became_active":"2024-04-26T20:21:38.422792+0000","last_became_peered":"2024-04-26T20:21:38.422792+0000","last_unstale":"2024-04-26T20:26:37.964864+0000","last_undegraded":"2024-04-26T20:26:37.964864+0000","last_fullsized":"2024-04-26T20:26:37.964864+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'119","last_scrub_stamp":"2024-04-26T20:26:37.964770+0000","last_deep_scrub":"51'119","last_deep_scrub_stamp":"2024-04-26T20:26:37.964770+0000","last_clean_scrub_stamp":"2024-04-26T20:26:37.964770+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:45.627702+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1204224,"data_stored":842332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":251964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574916,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1204224,"data_stored":842332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:26:58.287 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T20:26:58.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.386+0000 7ff5a2b46640 1 Processor -- start 2024-04-26T20:26:58.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.386+0000 7ff5a2b46640 1 -- start start 2024-04-26T20:26:58.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.387+0000 7ff5a2b46640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff59c160800 0x7ff59c1610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.387+0000 7ff5a2b46640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff59c0d0930 con 0x7ff59c0d2320 2024-04-26T20:26:58.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.387+0000 7ff5a2b46640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff59c0d0c00 con 0x7ff59c0d5b60 2024-04-26T20:26:58.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.387+0000 7ff5a2b46640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff59c0d0ee0 con 0x7ff59c160800 2024-04-26T20:26:58.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.387+0000 7ff5a10e3640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff59c0d2320 0x7ff59c0d3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45728/0 (socket says 172.21.15.37:45728) 2024-04-26T20:26:58.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.387+0000 7ff5a10e3640 1 -- 172.21.15.37:0/565221951 learned_addr learned my addr 172.21.15.37:0/565221951 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:58.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.388+0000 7ff5a08e2640 1 --2- 172.21.15.37:0/565221951 >> v2:172.21.15.134:3300/0 conn(0x7ff59c160800 0x7ff59c1610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.388+0000 7ff59b7fe640 1 -- 172.21.15.37:0/565221951 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1514265688 0 0) 0x7ff59c0d0930 con 0x7ff59c0d2320 2024-04-26T20:26:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.388+0000 7ff59b7fe640 1 -- 172.21.15.37:0/565221951 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff584003520 con 0x7ff59c0d2320 2024-04-26T20:26:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.388+0000 7ff5a08e2640 1 -- 172.21.15.37:0/565221951 >> v1:172.21.15.134:6789/0 conn(0x7ff59c0d5b60 legacy=0x7ff59c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.388+0000 7ff5a08e2640 1 -- 172.21.15.37:0/565221951 >> v1:172.21.15.37:6789/0 conn(0x7ff59c0d2320 legacy=0x7ff59c0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.388+0000 7ff5a08e2640 1 -- 172.21.15.37:0/565221951 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59c0d11c0 con 0x7ff59c160800 2024-04-26T20:26:58.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.389+0000 7ff5a08e2640 1 --2- 172.21.15.37:0/565221951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c160800 0x7ff59c1610c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff594009d90 tx=0x7ff59402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e754951b26e43ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.389+0000 7ff59b7fe640 1 -- 172.21.15.37:0/565221951 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594007740 con 0x7ff59c160800 2024-04-26T20:26:58.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.389+0000 7ff59b7fe640 1 -- 172.21.15.37:0/565221951 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5940078e0 con 0x7ff59c160800 2024-04-26T20:26:58.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.389+0000 7ff59b7fe640 1 -- 172.21.15.37:0/565221951 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594007cb0 con 0x7ff59c160800 2024-04-26T20:26:58.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.389+0000 7ff5a2b46640 1 -- 172.21.15.37:0/565221951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c160800 msgr2=0x7ff59c1610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.389+0000 7ff5a2b46640 1 --2- 172.21.15.37:0/565221951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c160800 0x7ff59c1610c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff594009d90 tx=0x7ff59402a660 comp rx=0 tx=0).stop 2024-04-26T20:26:58.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.389+0000 7ff5a2b46640 1 -- 172.21.15.37:0/565221951 shutdown_connections 2024-04-26T20:26:58.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.389+0000 7ff5a2b46640 1 --2- 172.21.15.37:0/565221951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c160800 0x7ff59c1610c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.389+0000 7ff5a2b46640 1 -- 172.21.15.37:0/565221951 >> 172.21.15.37:0/565221951 conn(0x7ff59c09e550 msgr2=0x7ff59c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:58.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.389+0000 7ff5a2b46640 1 -- 172.21.15.37:0/565221951 shutdown_connections 2024-04-26T20:26:58.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.389+0000 7ff5a2b46640 1 -- 172.21.15.37:0/565221951 wait complete. 2024-04-26T20:26:58.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.391+0000 7ff5a2b46640 1 Processor -- start 2024-04-26T20:26:58.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.391+0000 7ff5a2b46640 1 -- start start 2024-04-26T20:26:58.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a2b46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c0d2320 0x7ff59c157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a2b46640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff59c0d5b60 0x7ff59c0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a2b46640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c160800 0x7ff59c0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a2b46640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c0b6300 con 0x7ff59c0d2320 2024-04-26T20:26:58.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a2b46640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c0d30f0 con 0x7ff59c160800 2024-04-26T20:26:58.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a2b46640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff59c160be0 con 0x7ff59c0d5b60 2024-04-26T20:26:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a08e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c0d2320 0x7ff59c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a08e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c0d2320 0x7ff59c157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38828/0 (socket says 172.21.15.37:38828) 2024-04-26T20:26:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a08e2640 1 -- 172.21.15.37:0/1377514046 learned_addr learned my addr 172.21.15.37:0/1377514046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a10e3640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c160800 0x7ff59c0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff59bfff640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff59c0d5b60 0x7ff59c0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a08e2640 1 -- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff59c0d5b60 msgr2=0x7ff59c0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a08e2640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff59c0d5b60 0x7ff59c0c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a08e2640 1 -- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c160800 msgr2=0x7ff59c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a08e2640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c160800 0x7ff59c0ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a08e2640 1 -- 172.21.15.37:0/1377514046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59c0cd770 con 0x7ff59c0d2320 2024-04-26T20:26:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff59bfff640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff59c0d5b60 0x7ff59c0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a08e2640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c0d2320 0x7ff59c157770 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff594009d90 tx=0x7ff594004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5997fa640 1 -- 172.21.15.37:0/1377514046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59403f070 con 0x7ff59c0d2320 2024-04-26T20:26:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5997fa640 1 -- 172.21.15.37:0/1377514046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff594031c40 con 0x7ff59c0d2320 2024-04-26T20:26:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a2b46640 1 -- 172.21.15.37:0/1377514046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59c0b9520 con 0x7ff59c0d2320 2024-04-26T20:26:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5997fa640 1 -- 172.21.15.37:0/1377514046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594009450 con 0x7ff59c0d2320 2024-04-26T20:26:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.392+0000 7ff5a2b46640 1 -- 172.21.15.37:0/1377514046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff59c0b9ae0 con 0x7ff59c0d2320 2024-04-26T20:26:58.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.393+0000 7ff5a2b46640 1 -- 172.21.15.37:0/1377514046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff59c06a360 con 0x7ff59c0d2320 2024-04-26T20:26:58.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.394+0000 7ff5997fa640 1 -- 172.21.15.37:0/1377514046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7ff594043410 con 0x7ff59c0d2320 2024-04-26T20:26:58.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.394+0000 7ff5997fa640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff58806cbc0 0x7ff58806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.394+0000 7ff5997fa640 1 -- 172.21.15.37:0/1377514046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7ff5940ad200 con 0x7ff59c0d2320 2024-04-26T20:26:58.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.394+0000 7ff59bfff640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff58806cbc0 0x7ff58806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.395+0000 7ff59bfff640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff58806cbc0 0x7ff58806f080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff59c157cb0 tx=0x7ff590009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.399+0000 7ff5997fa640 1 -- 172.21.15.37:0/1377514046 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff59407b4d0 con 0x7ff59c0d2320 2024-04-26T20:26:58.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.583+0000 7ff5a2b46640 1 -- 172.21.15.37:0/1377514046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7ff59c0cf600 con 0x7ff59c0d2320 2024-04-26T20:26:58.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.586+0000 7ff5997fa640 1 -- 172.21.15.37:0/1377514046 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7ff59407add0 con 0x7ff59c0d2320 2024-04-26T20:26:58.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.587+0000 7ff5997fa640 1 -- 172.21.15.37:0/1377514046 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff5940abe50 con 0x7ff59c0d2320 2024-04-26T20:26:58.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 -- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff58806cbc0 msgr2=0x7ff58806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff58806cbc0 0x7ff58806f080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff59c157cb0 tx=0x7ff590009040 comp rx=0 tx=0).stop 2024-04-26T20:26:58.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 -- 172.21.15.37:0/1377514046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c0d2320 msgr2=0x7ff59c157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c0d2320 0x7ff59c157770 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff594009d90 tx=0x7ff594004a40 comp rx=0 tx=0).stop 2024-04-26T20:26:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 -- 172.21.15.37:0/1377514046 shutdown_connections 2024-04-26T20:26:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c160800 0x7ff59c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7ff58806cbc0 0x7ff58806f080 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff59c0d5b60 0x7ff59c0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 --2- 172.21.15.37:0/1377514046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c0d2320 0x7ff59c157770 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 -- 172.21.15.37:0/1377514046 >> 172.21.15.37:0/1377514046 conn(0x7ff59c09e550 msgr2=0x7ff59c0d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 -- 172.21.15.37:0/1377514046 shutdown_connections 2024-04-26T20:26:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.589+0000 7ff5a2b46640 1 -- 172.21.15.37:0/1377514046 wait complete. 2024-04-26T20:26:58.601 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:26:58.601 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:58.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.701+0000 7f1fa16c8640 1 Processor -- start 2024-04-26T20:26:58.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.701+0000 7f1fa16c8640 1 -- start start 2024-04-26T20:26:58.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1fa16c8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1f9c06e530 0x7f1f9c0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1fa16c8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f9c0cf640 0x7f1f9c0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1fa16c8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f9c0d2020 con 0x7f1f9c0d1c40 2024-04-26T20:26:58.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1fa16c8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f1f9c0ce4b0 con 0x7f1f9c06e530 2024-04-26T20:26:58.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1fa16c8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9c0cb850 con 0x7f1f9c0cf640 2024-04-26T20:26:58.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1f93fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1f9c06e530 0x7f1f9c0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1f9affd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1f9c0d1c40 0x7f1f9c06fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45744/0 (socket says 172.21.15.37:45744) 2024-04-26T20:26:58.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1f9affd640 1 -- 172.21.15.37:0/2510660521 learned_addr learned my addr 172.21.15.37:0/2510660521 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-26T20:26:58.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1f93fff640 1 -- 172.21.15.37:0/2510660521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f9c0cf640 msgr2=0x7f1f9c0cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:58.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1f93fff640 1 --2- 172.21.15.37:0/2510660521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f9c0cf640 0x7f1f9c0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1f93fff640 1 -- 172.21.15.37:0/2510660521 >> v1:172.21.15.37:6789/0 conn(0x7f1f9c0d1c40 legacy=0x7f1f9c06fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1f93fff640 1 -- 172.21.15.37:0/2510660521 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c0ce700 con 0x7f1f9c06e530 2024-04-26T20:26:58.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.702+0000 7f1f9a7fc640 1 -- 172.21.15.37:0/2510660521 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 420039974 0 0) 0x7f1f9c0d2020 con 0x7f1f9c0d1c40 2024-04-26T20:26:58.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.703+0000 7f1f93fff640 1 --2- 172.21.15.37:0/2510660521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f9c06e530 0x7f1f9c0cf100 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1f840097a0 tx=0x7f1f8402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae6003f789cee19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.703+0000 7f1f9a7fc640 1 -- 172.21.15.37:0/2510660521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84037070 con 0x7f1f9c06e530 2024-04-26T20:26:58.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.703+0000 7f1f9a7fc640 1 -- 172.21.15.37:0/2510660521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1f84032440 con 0x7f1f9c06e530 2024-04-26T20:26:58.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.703+0000 7f1f9a7fc640 1 -- 172.21.15.37:0/2510660521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84032830 con 0x7f1f9c06e530 2024-04-26T20:26:58.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.704+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2510660521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f9c06e530 msgr2=0x7f1f9c0cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.704+0000 7f1fa16c8640 1 --2- 172.21.15.37:0/2510660521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f9c06e530 0x7f1f9c0cf100 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1f840097a0 tx=0x7f1f8402a180 comp rx=0 tx=0).stop 2024-04-26T20:26:58.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.704+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2510660521 shutdown_connections 2024-04-26T20:26:58.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.704+0000 7f1fa16c8640 1 --2- 172.21.15.37:0/2510660521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f9c0cf640 0x7f1f9c0cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.704+0000 7f1fa16c8640 1 --2- 172.21.15.37:0/2510660521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f9c06e530 0x7f1f9c0cf100 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.704+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2510660521 >> 172.21.15.37:0/2510660521 conn(0x7f1f9c073880 msgr2=0x7f1f9c073c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:58.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.704+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2510660521 shutdown_connections 2024-04-26T20:26:58.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.704+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2510660521 wait complete. 2024-04-26T20:26:58.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.705+0000 7f1fa16c8640 1 Processor -- start 2024-04-26T20:26:58.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.705+0000 7f1fa16c8640 1 -- start start 2024-04-26T20:26:58.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1fa16c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f9c06e530 0x7f1f9c1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1fa16c8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f9c0cf640 0x7f1f9c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1fa16c8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f9c0d1c40 0x7f1f9c1f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1fa16c8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c06e990 con 0x7f1f9c06e530 2024-04-26T20:26:58.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1fa16c8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c0cbb30 con 0x7f1f9c0d1c40 2024-04-26T20:26:58.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1fa16c8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9c0cc500 con 0x7f1f9c0cf640 2024-04-26T20:26:58.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f9affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f9c06e530 0x7f1f9c1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f9affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f9c06e530 0x7f1f9c1591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38844/0 (socket says 172.21.15.37:38844) 2024-04-26T20:26:58.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f9affd640 1 -- 172.21.15.37:0/2966414392 learned_addr learned my addr 172.21.15.37:0/2966414392 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T20:26:58.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f9affd640 1 -- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f9c0cf640 msgr2=0x7f1f9c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f9affd640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f9c0cf640 0x7f1f9c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f9affd640 1 -- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f9c0d1c40 msgr2=0x7f1f9c1f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f9affd640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f9c0d1c40 0x7f1f9c1f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f9affd640 1 -- 172.21.15.37:0/2966414392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c1f5e70 con 0x7f1f9c06e530 2024-04-26T20:26:58.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f9affd640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f9c06e530 0x7f1f9c1591c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c430 tx=0x7f1f8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f937fe640 1 -- 172.21.15.37:0/2966414392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0074e0 con 0x7f1f9c06e530 2024-04-26T20:26:58.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f937fe640 1 -- 172.21.15.37:0/2966414392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1f8c0047e0 con 0x7f1f9c06e530 2024-04-26T20:26:58.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2966414392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c1597f0 con 0x7f1f9c06e530 2024-04-26T20:26:58.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1f937fe640 1 -- 172.21.15.37:0/2966414392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c004bf0 con 0x7f1f9c06e530 2024-04-26T20:26:58.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.706+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2966414392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c159db0 con 0x7f1f9c06e530 2024-04-26T20:26:58.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.707+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2966414392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f9c06a360 con 0x7f1f9c06e530 2024-04-26T20:26:58.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.708+0000 7f1f937fe640 1 -- 172.21.15.37:0/2966414392 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f1f8c011070 con 0x7f1f9c06e530 2024-04-26T20:26:58.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.708+0000 7f1f937fe640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1f7c06cbc0 0x7f1f7c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.708+0000 7f1f937fe640 1 -- 172.21.15.37:0/2966414392 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f1f8c08baf0 con 0x7f1f9c06e530 2024-04-26T20:26:58.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.709+0000 7f1f93fff640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1f7c06cbc0 0x7f1f7c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.709+0000 7f1f93fff640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1f7c06cbc0 0x7f1f7c06f080 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1f84034000 tx=0x7f1f8402c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.713+0000 7f1f937fe640 1 -- 172.21.15.37:0/2966414392 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f9c1f5e70 con 0x7f1f9c06e530 2024-04-26T20:26:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.936+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2966414392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f9c0d1460 con 0x7f1f9c06e530 2024-04-26T20:26:58.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.936+0000 7f1f937fe640 1 -- 172.21.15.37:0/2966414392 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+3866 (secure 0 0 0) 0x7f1f8c059d60 con 0x7f1f9c06e530 2024-04-26T20:26:58.937 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T20:26:58.938 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/689917778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":689917778},{"type":"v1","addr":"172.21.15.134:6836","nonce":689917778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5362,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/497972484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":497972484},{"type":"v1","addr":"172.21.15.134:6837","nonce":497972484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/221502372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":221502372},{"type":"v1","addr":"172.21.15.37:6833","nonce":221502372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":6051,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2231278586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2231278586},{"type":"v1","addr":"172.21.15.37:6835","nonce":2231278586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[]} 2024-04-26T20:26:58.938 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-26T20:26:58.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1f7c06cbc0 msgr2=0x7f1f7c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1f7c06cbc0 0x7f1f7c06f080 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1f84034000 tx=0x7f1f8402c000 comp rx=0 tx=0).stop 2024-04-26T20:26:58.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2966414392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f9c06e530 msgr2=0x7f1f9c1591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f9c06e530 0x7f1f9c1591c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c430 tx=0x7f1f8c00c8f0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2966414392 shutdown_connections 2024-04-26T20:26:58.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f9c0d1c40 0x7f1f9c1f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:6832/2709823969,v1:172.21.15.134:6833/2709823969] conn(0x7f1f7c06cbc0 0x7f1f7c06f080 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f9c0cf640 0x7f1f9c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 --2- 172.21.15.37:0/2966414392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f9c06e530 0x7f1f9c1591c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2966414392 >> 172.21.15.37:0/2966414392 conn(0x7f1f9c073880 msgr2=0x7f1f9c0ae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:58.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2966414392 shutdown_connections 2024-04-26T20:26:58.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T20:26:58.938+0000 7f1fa16c8640 1 -- 172.21.15.37:0/2966414392 wait complete. 2024-04-26T20:26:58.950 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T20:26:58.950 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T20:26:58.951 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:26:58.951 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:58.960 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:26:58.960 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:26:58.960 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.009 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:26:59.009 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:26:59.009 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.020 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:26:59.021 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:26:59.021 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.072 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:26:59.072 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T20:26:59.072 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T20:26:59.072 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.101 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T20:26:59.101 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T20:26:59.101 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.166 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T20:26:59.166 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T20:26:59.166 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.229 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T20:26:59.230 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T20:26:59.230 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.296 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T20:26:59.296 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T20:26:59.296 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.328 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T20:26:59.329 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T20:26:59.329 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.396 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T20:26:59.396 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T20:26:59.396 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.464 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T20:26:59.464 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T20:26:59.464 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.532 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T20:26:59.532 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T20:26:59.532 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T20:26:59.532 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.557 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T20:26:59.557 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T20:26:59.557 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.603 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T20:26:59.603 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T20:26:59.604 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T20:26:59.604 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.615 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T20:26:59.615 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T20:26:59.615 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.656 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T20:26:59.656 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T20:26:59.656 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:59.709 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T20:26:59.709 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T20:26:59.710 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-04-26T20:26:59.749 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:26:59.749 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T20:26:59.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:26:59.954 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T20:27:00.030 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:27:00.030 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T20:27:00.120 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:27:00.120 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T20:27:00.171 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:27:00.171 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T20:27:00.326 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:27:00.327 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T20:27:00.405 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:27:00.406 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T20:27:00.455 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:27:00.456 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T20:27:00.507 INFO:tasks.ceph:Archiving mon data... 2024-04-26T20:27:00.508 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165/data/mon.a.tgz 2024-04-26T20:27:00.508 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-04-26T20:27:00.524 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.39Vkf0dnXk 2024-04-26T20:27:00.525 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.39Vkf0dnXk 2024-04-26T20:27:00.683 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.39Vkf0dnXk 2024-04-26T20:27:00.766 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.39Vkf0dnXk is 462KB 2024-04-26T20:27:00.863 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.39Vkf0dnXk 2024-04-26T20:27:00.878 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165/data/mon.b.tgz 2024-04-26T20:27:00.878 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-04-26T20:27:00.896 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.D9S5tlFiVe 2024-04-26T20:27:00.897 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.D9S5tlFiVe 2024-04-26T20:27:01.112 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.D9S5tlFiVe 2024-04-26T20:27:01.194 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.D9S5tlFiVe is 628KB 2024-04-26T20:27:01.295 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.D9S5tlFiVe 2024-04-26T20:27:01.310 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165/data/mon.c.tgz 2024-04-26T20:27:01.310 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-04-26T20:27:01.363 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.FkwbzQRqYr 2024-04-26T20:27:01.364 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.FkwbzQRqYr 2024-04-26T20:27:01.541 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.FkwbzQRqYr 2024-04-26T20:27:01.622 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.FkwbzQRqYr is 628KB 2024-04-26T20:27:01.723 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.FkwbzQRqYr 2024-04-26T20:27:01.737 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T20:27:01.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T20:27:01.740 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T20:27:01.824 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T20:27:01.855 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T20:27:01.855 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T20:27:01.856 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165/remote/smithi037/crash 2024-04-26T20:27:01.857 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T20:27:01.891 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165/remote/smithi134/crash 2024-04-26T20:27:01.892 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T20:27:01.918 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T20:27:01.918 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:27:01.936 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:27:02.012 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T20:27:02.039 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T20:27:02.062 INFO:tasks.ceph:Compressing logs... 2024-04-26T20:27:02.062 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T20:27:02.082 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T20:27:02.105 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44623.log 2024-04-26T20:27:02.146 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34763.log 2024-04-26T20:27:02.146 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T20:27:02.146 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34894.log 2024-04-26T20:27:02.146 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34763.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34854.log 2024-04-26T20:27:02.146 INFO:teuthology.orchestra.run.smithi134.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34763.log.gz 2024-04-26T20:27:02.146 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34894.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T20:27:02.147 INFO:teuthology.orchestra.run.smithi134.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34894.log.gz 2024-04-26T20:27:02.147 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34814.log 2024-04-26T20:27:02.147 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph.audit.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.34854.log.gz 2024-04-26T20:27:02.147 INFO:teuthology.orchestra.run.smithi134.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T20:27:02.147 INFO:teuthology.orchestra.run.smithi134.stderr: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T20:27:02.147 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34814.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T20:27:02.147 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.7.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34814.log.gz 2024-04-26T20:27:02.147 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T20:27:02.148 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41524.log 2024-04-26T20:27:02.148 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T20:27:02.148 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.3.41524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41126.log 2024-04-26T20:27:02.148 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41524.log.gz 2024-04-26T20:27:02.148 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34683.log 2024-04-26T20:27:02.148 INFO:teuthology.orchestra.run.smithi134.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T20:27:02.148 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.1.41126.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T20:27:02.148 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41126.log.gz 2024-04-26T20:27:02.148 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34723.log 2024-04-26T20:27:02.149 INFO:teuthology.orchestra.run.smithi134.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T20:27:02.149 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mgr.x.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34683.log.gz 2024-04-26T20:27:02.149 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T20:27:02.149 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34974.log 2024-04-26T20:27:02.149 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mon.c.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34723.log.gz 2024-04-26T20:27:02.149 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T20:27:02.149 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42119.log 2024-04-26T20:27:02.149 INFO:teuthology.orchestra.run.smithi134.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34974.log.gz 2024-04-26T20:27:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42678.log 2024-04-26T20:27:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.1.42119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42119.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34934.log 2024-04-26T20:27:02.150 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-04-26T20:27:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T20:27:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.1.42678.log: /var/log/ceph/ceph-client.admin.34934.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42678.log.gz 2024-04-26T20:27:02.150 INFO:teuthology.orchestra.run.smithi134.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34934.log.gz 2024-04-26T20:27:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T20:27:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T20:27:02.248 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56973.log 2024-04-26T20:27:02.248 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61155.log 2024-04-26T20:27:02.248 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44623.log.gz 2024-04-26T20:27:02.248 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45127.log 2024-04-26T20:27:02.248 INFO:teuthology.orchestra.run.smithi037.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56973.log.gz 2024-04-26T20:27:02.249 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47628.log 2024-04-26T20:27:02.249 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46468.log 2024-04-26T20:27:02.249 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61155.log.gz 2024-04-26T20:27:02.249 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45127.log.gz 2024-04-26T20:27:02.249 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43016.log 2024-04-26T20:27:02.249 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47628.log.gz 2024-04-26T20:27:02.249 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48697.log 2024-04-26T20:27:02.249 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41643.log 2024-04-26T20:27:02.250 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43016.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46468.log.gz 2024-04-26T20:27:02.250 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43142.log 2024-04-26T20:27:02.250 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43016.log.gz 2024-04-26T20:27:02.250 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48697.log.gz 2024-04-26T20:27:02.250 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.41643.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.61342.log 2024-04-26T20:27:02.250 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41642.log 2024-04-26T20:27:02.250 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41643.log.gz 2024-04-26T20:27:02.250 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61342.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43142.log.gz 2024-04-26T20:27:02.251 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57924.log 2024-04-26T20:27:02.251 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61342.log.gz/var/log/ceph/ceph-client.admin.41642.log: 2024-04-26T20:27:02.251 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58765.log 2024-04-26T20:27:02.251 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41642.log.gz 2024-04-26T20:27:02.251 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45754.log 2024-04-26T20:27:02.251 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58765.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57924.log.gz 2024-04-26T20:27:02.251 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58849.log 2024-04-26T20:27:02.251 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45754.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58765.log.gz 2024-04-26T20:27:02.252 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56029.log 2024-04-26T20:27:02.252 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45754.log.gz 2024-04-26T20:27:02.252 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46216.log 2024-04-26T20:27:02.252 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58849.log: /var/log/ceph/ceph-client.admin.56029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45466.log 2024-04-26T20:27:02.252 INFO:teuthology.orchestra.run.smithi037.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.58849.log.gz 2024-04-26T20:27:02.252 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43322.log 2024-04-26T20:27:02.252 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56029.log.gz 2024-04-26T20:27:02.252 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46216.log.gz 2024-04-26T20:27:02.253 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55754.log 2024-04-26T20:27:02.253 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43322.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45466.log.gz 2024-04-26T20:27:02.253 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48154.log 2024-04-26T20:27:02.253 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43322.log.gz 2024-04-26T20:27:02.253 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58306.log 2024-04-26T20:27:02.253 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55754.log.gz 2024-04-26T20:27:02.253 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46865.log 2024-04-26T20:27:02.253 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48154.log.gz 2024-04-26T20:27:02.253 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56401.log 2024-04-26T20:27:02.254 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58306.log: /var/log/ceph/ceph-client.admin.46865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59758.log 2024-04-26T20:27:02.254 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58306.log.gz 2024-04-26T20:27:02.254 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57966.log 88.2% 2024-04-26T20:27:02.254 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46865.log.gz 2024-04-26T20:27:02.254 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56401.log.gz 2024-04-26T20:27:02.254 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.59758.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59364.log 2024-04-26T20:27:02.254 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57966.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59758.log.gz 2024-04-26T20:27:02.255 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43184.log 2024-04-26T20:27:02.255 INFO:teuthology.orchestra.run.smithi037.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.57966.log.gz 2024-04-26T20:27:02.255 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47405.log 2024-04-26T20:27:02.255 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43184.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59364.log.gz 2024-04-26T20:27:02.255 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40755.log 2024-04-26T20:27:02.255 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41977.log 2024-04-26T20:27:02.255 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43184.log.gz 2024-04-26T20:27:02.255 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47405.log.gz 2024-04-26T20:27:02.256 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59632.log 2024-04-26T20:27:02.256 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56711.log 89.5% 2024-04-26T20:27:02.256 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40755.log.gz 2024-04-26T20:27:02.256 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59632.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41977.log.gz 2024-04-26T20:27:02.256 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46510.log 2024-04-26T20:27:02.256 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59632.log.gz 2024-04-26T20:27:02.256 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45424.log 2024-04-26T20:27:02.256 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46510.log: /var/log/ceph/ceph-client.admin.48529.log 2024-04-26T20:27:02.256 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56711.log.gz 2024-04-26T20:27:02.257 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57108.log 2024-04-26T20:27:02.257 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46510.log.gz 2024-04-26T20:27:02.257 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48529.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45424.log.gz 2024-04-26T20:27:02.257 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41976.log 2024-04-26T20:27:02.257 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57108.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48529.log.gz 2024-04-26T20:27:02.257 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42310.log 2024-04-26T20:27:02.257 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41976.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57108.log.gz 2024-04-26T20:27:02.257 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T20:27:02.258 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41976.log.gz 2024-04-26T20:27:02.258 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60903.log 2024-04-26T20:27:02.258 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42310.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60429.log 2024-04-26T20:27:02.258 INFO:teuthology.orchestra.run.smithi037.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42310.log.gz 2024-04-26T20:27:02.258 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57318.log 2024-04-26T20:27:02.258 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60903.log.gz 2024-04-26T20:27:02.258 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45670.log 2024-04-26T20:27:02.258 INFO:teuthology.orchestra.run.smithi037.stderr: 85.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T20:27:02.259 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4%/var/log/ceph/ceph-client.admin.57318.log: -- replaced with /var/log/ceph/ceph-client.admin.60429.log.gz 2024-04-26T20:27:02.259 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55399.log 2024-04-26T20:27:02.259 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40839.log 2024-04-26T20:27:02.259 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57318.log.gz 2024-04-26T20:27:02.259 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45670.log.gz 2024-04-26T20:27:02.259 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57743.log 2024-04-26T20:27:02.259 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55399.log.gz 2024-04-26T20:27:02.259 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48613.log 2024-04-26T20:27:02.259 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57743.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.40839.log.gz 2024-04-26T20:27:02.260 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41973.log 2024-04-26T20:27:02.260 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57743.log.gz 2024-04-26T20:27:02.260 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46012.log 2024-04-26T20:27:02.260 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42512.log 2024-04-26T20:27:02.260 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48613.log.gz 2024-04-26T20:27:02.260 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42313.log 2024-04-26T20:27:02.260 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41973.log.gz 2024-04-26T20:27:02.260 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46012.log.gz 2024-04-26T20:27:02.261 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46594.log 2024-04-26T20:27:02.261 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56197.log 2024-04-26T20:27:02.261 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42512.log.gz 2024-04-26T20:27:02.261 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46594.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42313.log.gz 2024-04-26T20:27:02.261 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41635.log 2024-04-26T20:27:02.261 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46594.log.gz 2024-04-26T20:27:02.261 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60987.log 2024-04-26T20:27:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56197.log: /var/log/ceph/ceph-client.admin.41635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59406.log 2024-04-26T20:27:02.262 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56197.log.gz 2024-04-26T20:27:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60987.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T20:27:02.262 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41635.log.gz 2024-04-26T20:27:02.262 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60987.log.gz 2024-04-26T20:27:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47026.log 2024-04-26T20:27:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.2.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.59406.log.gz -5 2024-04-26T20:27:02.262 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58096.log 2024-04-26T20:27:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59548.log 2024-04-26T20:27:02.263 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47026.log.gz 2024-04-26T20:27:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.58096.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56071.log 2024-04-26T20:27:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5/var/log/ceph/ceph-client.admin.59548.log: --verbose -- /var/log/ceph/ceph-client.admin.57276.log 2024-04-26T20:27:02.263 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58096.log.gz 2024-04-26T20:27:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56071.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59548.log.gz 2024-04-26T20:27:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56753.log 2024-04-26T20:27:02.263 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56071.log.gz 2024-04-26T20:27:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40370.log 2024-04-26T20:27:02.264 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57276.log.gz 2024-04-26T20:27:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57234.log 2024-04-26T20:27:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40370.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56753.log.gz 2024-04-26T20:27:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47968.log 2024-04-26T20:27:02.264 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40370.log.gz 2024-04-26T20:27:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5/var/log/ceph/ceph-client.admin.57234.log: --verbose -- /var/log/ceph/ceph-client.admin.40628.log 2024-04-26T20:27:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41968.log 2024-04-26T20:27:02.264 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57234.log.gz 2024-04-26T20:27:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40628.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47968.log.gz 2024-04-26T20:27:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T20:27:02.265 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40628.log.gz 2024-04-26T20:27:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5/var/log/ceph/ceph-client.admin.41968.log: --verbose -- /var/log/ceph/ceph-client.admin.42764.log 2024-04-26T20:27:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.audit.log: gzip 88.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41968.log.gz -- 2024-04-26T20:27:02.265 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.34387.log 2024-04-26T20:27:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47884.log 2024-04-26T20:27:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34387.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42764.log.gz 2024-04-26T20:27:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46258.log 2024-04-26T20:27:02.266 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34387.log.gz 2024-04-26T20:27:02.266 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60113.log 2024-04-26T20:27:02.266 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44959.log 2024-04-26T20:27:02.266 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47884.log.gz 2024-04-26T20:27:02.266 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46258.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T20:27:02.266 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60113.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46258.log.gz 2024-04-26T20:27:02.266 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42305.log 2024-04-26T20:27:02.266 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60113.log.gz 2024-04-26T20:27:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43100.log 2024-04-26T20:27:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42305.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44959.log.gz 2024-04-26T20:27:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60155.log 2024-04-26T20:27:02.267 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42305.log.gz 2024-04-26T20:27:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40451.log 2024-04-26T20:27:02.267 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43100.log.gz 2024-04-26T20:27:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61300.log 2024-04-26T20:27:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60155.log: /var/log/ceph/ceph-client.admin.40451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61260.log 2024-04-26T20:27:02.267 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60155.log.gz 2024-04-26T20:27:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61300.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40451.log.gz 2024-04-26T20:27:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56317.log 2024-04-26T20:27:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61260.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61300.log.gz 2024-04-26T20:27:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59674.log 2024-04-26T20:27:02.268 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61260.log.gz 2024-04-26T20:27:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44749.log 2024-04-26T20:27:02.268 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56317.log.gz 2024-04-26T20:27:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46342.log 2024-04-26T20:27:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57360.log 2024-04-26T20:27:02.269 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59674.log.gz 2024-04-26T20:27:02.269 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44749.log.gz 2024-04-26T20:27:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44413.log 2024-04-26T20:27:02.269 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46342.log.gz 2024-04-26T20:27:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44103.log 2024-04-26T20:27:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44329.log 2024-04-26T20:27:02.269 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57360.log.gz 2024-04-26T20:27:02.269 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44413.log.gz 2024-04-26T20:27:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.2.44103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44103.log.gz 2024-04-26T20:27:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47363.log 2024-04-26T20:27:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40670.log 2024-04-26T20:27:02.270 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2%/var/log/ceph/ceph-client.admin.47363.log: -- replaced with /var/log/ceph/ceph-client.admin.44329.log.gz 2024-04-26T20:27:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55544.log 2024-04-26T20:27:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57150.log 2024-04-26T20:27:02.270 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47363.log.gz 2024-04-26T20:27:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57535.log 2024-04-26T20:27:02.271 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40670.log.gz 2024-04-26T20:27:02.271 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57150.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55544.log.gz 2024-04-26T20:27:02.271 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56113.log 2024-04-26T20:27:02.271 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57150.log.gz 2024-04-26T20:27:02.271 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57535.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T20:27:02.271 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46945.log 2024-04-26T20:27:02.271 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57535.log.gz 2024-04-26T20:27:02.272 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.3.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56113.log.gz 2024-04-26T20:27:02.272 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46132.log 2024-04-26T20:27:02.272 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43795.log 2024-04-26T20:27:02.272 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46945.log.gz 2024-04-26T20:27:02.272 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48655.log 2024-04-26T20:27:02.272 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58414.log 2024-04-26T20:27:02.272 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43795.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46132.log.gz 2024-04-26T20:27:02.272 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47068.log 2024-04-26T20:27:02.273 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43795.log.gz 2024-04-26T20:27:02.273 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48655.log.gz 2024-04-26T20:27:02.273 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46823.log 2024-04-26T20:27:02.273 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58414.log: /var/log/ceph/ceph-client.admin.47068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57192.log 2024-04-26T20:27:02.273 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58414.log.gz 2024-04-26T20:27:02.273 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46823.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47068.log.gz 2024-04-26T20:27:02.273 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59842.log 2024-04-26T20:27:02.273 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46823.log.gz 2024-04-26T20:27:02.274 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60566.log 2024-04-26T20:27:02.274 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57192.log.gz 2024-04-26T20:27:02.274 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58681.log 2024-04-26T20:27:02.274 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59842.log: /var/log/ceph/ceph-client.admin.60566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56585.log 2024-04-26T20:27:02.274 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59842.log.gz 2024-04-26T20:27:02.274 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58681.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60566.log.gz 2024-04-26T20:27:02.274 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59884.log 2024-04-26T20:27:02.275 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58681.log.gz 2024-04-26T20:27:02.275 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45169.log 2024-04-26T20:27:02.275 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56585.log: /var/log/ceph/ceph-client.admin.59884.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56585.log.gz 2024-04-26T20:27:02.275 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56795.log 2024-04-26T20:27:02.275 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59884.log.gz 2024-04-26T20:27:02.275 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57024.log 2024-04-26T20:27:02.275 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45169.log: /var/log/ceph/ceph-client.admin.56795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58008.log 2024-04-26T20:27:02.275 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45169.log.gz 2024-04-26T20:27:02.276 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57024.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56795.log.gz 2024-04-26T20:27:02.276 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57455.log 2024-04-26T20:27:02.276 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57024.log.gz 2024-04-26T20:27:02.276 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48739.log 2024-04-26T20:27:02.276 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56275.log 2024-04-26T20:27:02.276 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57455.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58008.log.gz 2024-04-26T20:27:02.276 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48739.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T20:27:02.276 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57455.log.gz 2024-04-26T20:27:02.276 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48739.log.gz 2024-04-26T20:27:02.277 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58180.log 2024-04-26T20:27:02.277 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56275.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60471.log 2024-04-26T20:27:02.277 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56275.log.gz 2024-04-26T20:27:02.277 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46090.log 2024-04-26T20:27:02.277 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58180.log.gz 2024-04-26T20:27:02.277 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55880.log 2024-04-26T20:27:02.277 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46090.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60471.log.gz 2024-04-26T20:27:02.277 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42554.log 2024-04-26T20:27:02.278 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46090.log.gz 2024-04-26T20:27:02.278 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41641.log 2024-04-26T20:27:02.278 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46636.log 2024-04-26T20:27:02.278 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55880.log.gz 2024-04-26T20:27:02.278 INFO:teuthology.orchestra.run.smithi037.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T20:27:02.278 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42554.log.gz 2024-04-26T20:27:02.278 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42596.log 2024-04-26T20:27:02.278 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41641.log.gz 2024-04-26T20:27:02.279 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47447.log 2024-04-26T20:27:02.279 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46636.log: /var/log/ceph/ceph-client.admin.42596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58807.log 2024-04-26T20:27:02.279 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46636.log.gz 2024-04-26T20:27:02.279 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42596.log.gz 2024-04-26T20:27:02.279 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45001.log 2024-04-26T20:27:02.279 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47195.log 2024-04-26T20:27:02.279 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58807.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47447.log.gz 2024-04-26T20:27:02.279 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60345.log 2024-04-26T20:27:02.279 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58807.log.gz 2024-04-26T20:27:02.280 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45001.log.gz 2024-04-26T20:27:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T20:27:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47195.log: /var/log/ceph/ceph-client.admin.60345.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T20:27:02.280 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47195.log.gz 2024-04-26T20:27:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.1.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60345.log.gz 2024-04-26T20:27:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59506.log 2024-04-26T20:27:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57658.log 2024-04-26T20:27:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.59506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44833.log 2024-04-26T20:27:02.281 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59506.log.gz 2024-04-26T20:27:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59058.log 2024-04-26T20:27:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.44833.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41294.log 2024-04-26T20:27:02.281 INFO:teuthology.orchestra.run.smithi037.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57658.log.gz 2024-04-26T20:27:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59058.log: gzip -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44833.log.gz 2024-04-26T20:27:02.281 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45970.log 2024-04-26T20:27:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41294.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59058.log.gz 2024-04-26T20:27:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T20:27:02.281 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41294.log.gz 2024-04-26T20:27:02.282 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56359.log 2024-04-26T20:27:02.282 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42974.log 2024-04-26T20:27:02.282 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45970.log.gz 2024-04-26T20:27:02.282 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60071.log 2024-04-26T20:27:02.282 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44665.log 2024-04-26T20:27:02.282 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56359.log.gz 2024-04-26T20:27:02.282 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42974.log: /var/log/ceph/ceph-client.admin.60071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58054.log 2024-04-26T20:27:02.282 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42974.log.gz 2024-04-26T20:27:02.283 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44665.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60071.log.gz 2024-04-26T20:27:02.283 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40969.log 2024-04-26T20:27:02.283 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47586.log 2024-04-26T20:27:02.283 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44665.log.gz 2024-04-26T20:27:02.283 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58054.log: /var/log/ceph/ceph-client.admin.40969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57785.log 2024-04-26T20:27:02.283 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58054.log.gz 2024-04-26T20:27:02.283 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40969.log.gz 2024-04-26T20:27:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55670.log 2024-04-26T20:27:02.284 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47586.log.gz 2024-04-26T20:27:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60693.log 2024-04-26T20:27:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57785.log: /var/log/ceph/ceph-client.admin.55670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45043.log 2024-04-26T20:27:02.284 INFO:teuthology.orchestra.run.smithi037.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T20:27:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60693.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57785.log.gz 2024-04-26T20:27:02.284 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55670.log.gz 2024-04-26T20:27:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42308.log 2024-04-26T20:27:02.284 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60693.log.gz 2024-04-26T20:27:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42638.log 2024-04-26T20:27:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45043.log: /var/log/ceph/ceph-client.admin.42308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60819.log 2024-04-26T20:27:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42806.log 2024-04-26T20:27:02.285 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42308.log.gz 2024-04-26T20:27:02.285 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45043.log.gz 2024-04-26T20:27:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60819.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42638.log.gz 2024-04-26T20:27:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45586.log 2024-04-26T20:27:02.285 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60819.log.gz 2024-04-26T20:27:02.286 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41640.log 2024-04-26T20:27:02.286 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42806.log: /var/log/ceph/ceph-client.admin.45586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46720.log 2024-04-26T20:27:02.286 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41972.log 2024-04-26T20:27:02.286 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45586.log.gz 2024-04-26T20:27:02.286 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42806.log.gz 2024-04-26T20:27:02.286 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46720.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41640.log.gz 2024-04-26T20:27:02.286 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46426.log 2024-04-26T20:27:02.286 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46720.log.gz 2024-04-26T20:27:02.287 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41980.log 2024-04-26T20:27:02.287 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41972.log: /var/log/ceph/ceph-client.admin.46426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57616.log 2024-04-26T20:27:02.287 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42848.log 2024-04-26T20:27:02.287 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46426.log.gz 2024-04-26T20:27:02.287 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41972.log.gz 2024-04-26T20:27:02.287 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57616.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41980.log.gz 2024-04-26T20:27:02.287 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61197.log 2024-04-26T20:27:02.287 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57616.log.gz 2024-04-26T20:27:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41168.log 2024-04-26T20:27:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42848.log: /var/log/ceph/ceph-client.admin.61197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41210.log 2024-04-26T20:27:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44497.log 2024-04-26T20:27:02.288 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61197.log.gz 2024-04-26T20:27:02.288 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42848.log.gz 2024-04-26T20:27:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41210.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41168.log.gz 2024-04-26T20:27:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45628.log 2024-04-26T20:27:02.288 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41210.log.gz 2024-04-26T20:27:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47279.log 2024-04-26T20:27:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44497.log: gzip/var/log/ceph/ceph-client.admin.45628.log: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T20:27:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47279.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45628.log.gz 2024-04-26T20:27:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47321.log 2024-04-26T20:27:02.289 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44497.log.gz 2024-04-26T20:27:02.289 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47279.log.gz 2024-04-26T20:27:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45382.log 2024-04-26T20:27:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57066.log 2024-04-26T20:27:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47321.log: /var/log/ceph/ceph-client.admin.45382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40881.log 2024-04-26T20:27:02.290 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57066.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47321.log.gz 2024-04-26T20:27:02.290 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45382.log.gz 2024-04-26T20:27:02.290 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60387.log 2024-04-26T20:27:02.290 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57066.log.gz 2024-04-26T20:27:02.290 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48112.log 2024-04-26T20:27:02.290 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40881.log: /var/log/ceph/ceph-client.admin.60387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40797.log 2024-04-26T20:27:02.290 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48112.log: 88.0% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40881.log.gz 2024-04-26T20:27:02.290 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60387.log.gz 2024-04-26T20:27:02.291 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.48112.log.gz 2024-04-26T20:27:02.291 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34648.log 2024-04-26T20:27:02.291 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46174.log 2024-04-26T20:27:02.291 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40797.log: /var/log/ceph/ceph-client.admin.34648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55712.log 2024-04-26T20:27:02.291 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46174.log: 87.1% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40797.log.gz 2024-04-26T20:27:02.291 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34648.log.gz 2024-04-26T20:27:02.291 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46174.log.gz 2024-04-26T20:27:02.291 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44917.log 2024-04-26T20:27:02.292 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45712.log 2024-04-26T20:27:02.292 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55712.log: /var/log/ceph/ceph-client.admin.44917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43058.log 2024-04-26T20:27:02.292 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45712.log: 88.7% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55712.log.gz 2024-04-26T20:27:02.292 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44917.log.gz 2024-04-26T20:27:02.292 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45712.log.gz 2024-04-26T20:27:02.292 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45844.log 2024-04-26T20:27:02.292 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56543.log 2024-04-26T20:27:02.292 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43058.log: /var/log/ceph/ceph-client.admin.45844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57700.log 2024-04-26T20:27:02.292 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56543.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43058.log.gz 2024-04-26T20:27:02.293 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45844.log.gz 2024-04-26T20:27:02.293 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41336.log 2024-04-26T20:27:02.293 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56543.log.gz 2024-04-26T20:27:02.293 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59160.log 2024-04-26T20:27:02.293 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57700.log: /var/log/ceph/ceph-client.admin.41336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56155.log 2024-04-26T20:27:02.293 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59160.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57700.log.gz 2024-04-26T20:27:02.293 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41336.log.gz 2024-04-26T20:27:02.294 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47491.log 2024-04-26T20:27:02.294 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59160.log.gz 2024-04-26T20:27:02.294 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44244.log 2024-04-26T20:27:02.294 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56155.log: /var/log/ceph/ceph-client.admin.47491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59202.log 2024-04-26T20:27:02.294 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44244.log: 89.2% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56155.log.gz 2024-04-26T20:27:02.294 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47491.log.gz 2024-04-26T20:27:02.294 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59800.log 2024-04-26T20:27:02.295 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44244.log.gz 2024-04-26T20:27:02.295 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60524.log 2024-04-26T20:27:02.295 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59202.log: gzip/var/log/ceph/ceph-client.admin.59800.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60651.log 2024-04-26T20:27:02.295 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60524.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59800.log.gz 2024-04-26T20:27:02.295 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59202.log.gz 2024-04-26T20:27:02.295 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60524.log.gz 2024-04-26T20:27:02.295 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57829.log 2024-04-26T20:27:02.295 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59100.log 2024-04-26T20:27:02.296 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60651.log.gz/var/log/ceph/ceph-client.admin.57829.log: 2024-04-26T20:27:02.296 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43660.log 2024-04-26T20:27:02.296 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57829.log.gz 2024-04-26T20:27:02.296 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46300.log 2024-04-26T20:27:02.296 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59100.log: /var/log/ceph/ceph-client.0.43660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48571.log 2024-04-26T20:27:02.296 INFO:teuthology.orchestra.run.smithi037.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43660.log.gz 2024-04-26T20:27:02.296 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46300.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59100.log.gz 2024-04-26T20:27:02.296 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48010.log 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46300.log.gz 2024-04-26T20:27:02.296 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-26T20:27:02.297 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44455.log 2024-04-26T20:27:02.297 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48571.log.gz/var/log/ceph/ceph-client.admin.48010.log: 2024-04-26T20:27:02.297 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40547.log 2024-04-26T20:27:02.297 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48010.log.gz 2024-04-26T20:27:02.297 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58264.log 2024-04-26T20:27:02.297 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44455.log: /var/log/ceph/ceph-client.admin.40547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42890.log 2024-04-26T20:27:02.297 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58264.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44455.log.gz 2024-04-26T20:27:02.297 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40547.log.gz 2024-04-26T20:27:02.298 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47758.log 2024-04-26T20:27:02.298 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58264.log.gz 2024-04-26T20:27:02.298 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48262.log 2024-04-26T20:27:02.298 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42890.log: /var/log/ceph/ceph-client.admin.47758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48070.log 2024-04-26T20:27:02.298 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48262.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47758.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42890.log.gz 2024-04-26T20:27:02.298 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-26T20:27:02.298 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48262.log.gz 2024-04-26T20:27:02.298 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47237.log 2024-04-26T20:27:02.299 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48196.log 2024-04-26T20:27:02.299 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48070.log: /var/log/ceph/ceph-client.admin.47237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56879.log 2024-04-26T20:27:02.299 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48196.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48070.log.gz 2024-04-26T20:27:02.299 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47237.log.gz 2024-04-26T20:27:02.299 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41126.log 2024-04-26T20:27:02.299 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41979.log 2024-04-26T20:27:02.299 INFO:teuthology.orchestra.run.smithi037.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.48196.log.gz 2024-04-26T20:27:02.299 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56879.log: /var/log/ceph/ceph-client.admin.41126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58138.log 2024-04-26T20:27:02.300 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56879.log.gz 2024-04-26T20:27:02.300 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7%/var/log/ceph/ceph-client.admin.41979.log: -- replaced with /var/log/ceph/ceph-client.admin.41126.log.gz 2024-04-26T20:27:02.300 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41644.log 2024-04-26T20:27:02.300 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41979.log.gz 2024-04-26T20:27:02.300 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41084.log 2024-04-26T20:27:02.300 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58138.log: /var/log/ceph/ceph-client.admin.41644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41420.log 2024-04-26T20:27:02.300 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41084.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58138.log.gz 2024-04-26T20:27:02.300 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41644.log.gz 2024-04-26T20:27:02.300 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44287.log 2024-04-26T20:27:02.300 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41084.log.gz 2024-04-26T20:27:02.301 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42176.log 2024-04-26T20:27:02.301 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41420.log: /var/log/ceph/ceph-client.admin.44287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56501.log 2024-04-26T20:27:02.301 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42176.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41420.log.gz 2024-04-26T20:27:02.301 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44287.log.gz 2024-04-26T20:27:02.301 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56627.log 2024-04-26T20:27:02.301 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42176.log.gz 2024-04-26T20:27:02.301 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59978.log 2024-04-26T20:27:02.301 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56501.log: /var/log/ceph/ceph-client.admin.56627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46678.log 2024-04-26T20:27:02.302 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59978.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56627.log.gz 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56501.log.gz 2024-04-26T20:27:02.302 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-26T20:27:02.302 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59978.log.gz 2024-04-26T20:27:02.302 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55796.log 2024-04-26T20:27:02.302 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45886.log 2024-04-26T20:27:02.302 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46678.log: /var/log/ceph/ceph-client.admin.55796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47800.log 2024-04-26T20:27:02.302 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45886.log: 88.6% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46678.log.gz 2024-04-26T20:27:02.302 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55796.log.gz 2024-04-26T20:27:02.303 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45886.log.gz 2024-04-26T20:27:02.303 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59448.log 2024-04-26T20:27:02.303 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44791.log 2024-04-26T20:27:02.303 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47800.log: gzip/var/log/ceph/ceph-client.admin.59448.log: -5 --verbose -- /var/log/ceph/ceph-client.0.48450.log 2024-04-26T20:27:02.303 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44791.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59448.log.gz 2024-04-26T20:27:02.303 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47800.log.gz 2024-04-26T20:27:02.303 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44791.log.gz 2024-04-26T20:27:02.303 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28356.log 2024-04-26T20:27:02.303 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.48450.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48450.log.gz 2024-04-26T20:27:02.304 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59280.log 2024-04-26T20:27:02.304 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.tmp-client.admin.28356.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28356.log.gz 2024-04-26T20:27:02.304 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60029.log 2024-04-26T20:27:02.304 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61071.log 2024-04-26T20:27:02.304 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59280.log.gz 2024-04-26T20:27:02.304 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46384.log 2024-04-26T20:27:02.304 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60029.log.gz 2024-04-26T20:27:02.304 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47716.log 2024-04-26T20:27:02.304 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61071.log: /var/log/ceph/ceph-client.admin.46384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56837.log 2024-04-26T20:27:02.305 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47716.log: 88.4% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46384.log.gz 2024-04-26T20:27:02.305 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61071.log.gz 2024-04-26T20:27:02.305 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42309.log 2024-04-26T20:27:02.305 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47716.log.gz 2024-04-26T20:27:02.305 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40925.log 2024-04-26T20:27:02.305 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55586.log 2024-04-26T20:27:02.305 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56837.log.gz 2024-04-26T20:27:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42309.log: /var/log/ceph/ceph-client.admin.40925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44371.log 2024-04-26T20:27:02.306 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42309.log.gz 2024-04-26T20:27:02.306 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2%/var/log/ceph/ceph-client.admin.55586.log: -- replaced with /var/log/ceph/ceph-client.admin.40925.log.gz 2024-04-26T20:27:02.306 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55586.log.gz 2024-04-26T20:27:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47110.log 2024-04-26T20:27:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45229.log 2024-04-26T20:27:02.306 INFO:teuthology.orchestra.run.smithi037.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.44371.log.gz 2024-04-26T20:27:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41638.log 2024-04-26T20:27:02.307 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47110.log.gz 2024-04-26T20:27:02.307 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45508.log 2024-04-26T20:27:02.307 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45229.log.gz 2024-04-26T20:27:02.307 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61113.log 2024-04-26T20:27:02.307 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41638.log: /var/log/ceph/ceph-client.admin.45508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58348.log 2024-04-26T20:27:02.307 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.61113.log: 89.4% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41638.log.gz 2024-04-26T20:27:02.307 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45508.log.gz 2024-04-26T20:27:02.307 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61113.log.gz 2024-04-26T20:27:02.307 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44875.log 2024-04-26T20:27:02.308 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47926.log 2024-04-26T20:27:02.308 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58348.log: gzip/var/log/ceph/ceph-client.admin.44875.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60735.log 2024-04-26T20:27:02.308 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47926.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44875.log.gz 2024-04-26T20:27:02.308 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47926.log.gz 2024-04-26T20:27:02.308 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59926.log 2024-04-26T20:27:02.308 INFO:teuthology.orchestra.run.smithi037.stderr: 91.5%/var/log/ceph/ceph-client.admin.60735.log: -- replaced with /var/log/ceph/ceph-client.admin.58348.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41978.log 2024-04-26T20:27:02.308 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-26T20:27:02.308 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60735.log.gz 2024-04-26T20:27:02.308 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59926.log: gzip -5 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59926.log.gz 2024-04-26T20:27:02.309 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41252.log 2024-04-26T20:27:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47535.log 88.6% 2024-04-26T20:27:02.309 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41978.log.gz 2024-04-26T20:27:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61029.log 2024-04-26T20:27:02.309 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41252.log.gz 2024-04-26T20:27:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55483.log 2024-04-26T20:27:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47535.log: /var/log/ceph/ceph-client.admin.61029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60861.log 2024-04-26T20:27:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55483.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47535.log.gz 2024-04-26T20:27:02.309 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61029.log.gz 2024-04-26T20:27:02.309 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55483.log.gz 2024-04-26T20:27:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56921.log 2024-04-26T20:27:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41038.log 2024-04-26T20:27:02.310 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60861.log.gz/var/log/ceph/ceph-client.admin.56921.log: 2024-04-26T20:27:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47153.log 2024-04-26T20:27:02.310 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56921.log.gz 2024-04-26T20:27:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55441.log 2024-04-26T20:27:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41038.log: /var/log/ceph/ceph-client.admin.47153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59590.log 2024-04-26T20:27:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55441.log: 88.0% 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47153.log.gz 2024-04-26T20:27:02.311 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41038.log.gz 2024-04-26T20:27:02.311 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56669.log 2024-04-26T20:27:02.311 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55441.log.gz 2024-04-26T20:27:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59590.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T20:27:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44581.log 2024-04-26T20:27:02.311 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59590.log.gz 2024-04-26T20:27:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47670.log 2024-04-26T20:27:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mds.a.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56669.log.gz 2024-04-26T20:27:02.312 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44581.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44581.log.gz 2024-04-26T20:27:02.312 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59322.log 2024-04-26T20:27:02.312 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41378.log 2024-04-26T20:27:02.312 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47670.log.gz 2024-04-26T20:27:02.312 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56443.log 2024-04-26T20:27:02.312 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41378.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59322.log.gz 2024-04-26T20:27:02.312 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41378.log.gz 2024-04-26T20:27:02.312 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42722.log 2024-04-26T20:27:02.312 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41631.log 2024-04-26T20:27:02.313 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56443.log.gz 2024-04-26T20:27:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40713.log 2024-04-26T20:27:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41631.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42722.log.gz 2024-04-26T20:27:02.313 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41631.log.gz 2024-04-26T20:27:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46778.log 2024-04-26T20:27:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46552.log 2024-04-26T20:27:02.313 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40713.log.gz 2024-04-26T20:27:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58222.log 2024-04-26T20:27:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46552.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46778.log.gz 2024-04-26T20:27:02.314 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46552.log.gz 2024-04-26T20:27:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41462.log 2024-04-26T20:27:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60777.log 2024-04-26T20:27:02.314 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58222.log.gz 2024-04-26T20:27:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45085.log 2024-04-26T20:27:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60777.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41462.log.gz 2024-04-26T20:27:02.315 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60777.log.gz 2024-04-26T20:27:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44707.log 2024-04-26T20:27:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57873.log 2024-04-26T20:27:02.315 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45085.log.gz/var/log/ceph/ceph-client.admin.44707.log: 2024-04-26T20:27:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42932.log 2024-04-26T20:27:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57873.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44707.log.gz 2024-04-26T20:27:02.315 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57873.log.gz 2024-04-26T20:27:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45340.log 2024-04-26T20:27:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45928.log 2024-04-26T20:27:02.316 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42932.log.gz 2024-04-26T20:27:02.316 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34752.log 2024-04-26T20:27:02.316 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45928.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45340.log.gz 2024-04-26T20:27:02.316 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45928.log.gz 2024-04-26T20:27:02.316 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42316.log 2024-04-26T20:27:02.316 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55838.log 2024-04-26T20:27:02.316 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34752.log.gz 2024-04-26T20:27:02.316 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58723.log 2024-04-26T20:27:02.316 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55838.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42316.log.gz 2024-04-26T20:27:02.317 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55838.log.gz 2024-04-26T20:27:02.317 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47842.log 2024-04-26T20:27:02.317 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.58723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44539.log 2024-04-26T20:27:02.317 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58723.log.gz 2024-04-26T20:27:02.317 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60609.log 2024-04-26T20:27:02.317 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44539.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47842.log.gz 2024-04-26T20:27:02.317 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44539.log.gz 2024-04-26T20:27:02.317 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42680.log 2024-04-26T20:27:02.318 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.60609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42315.log 2024-04-26T20:27:02.318 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60609.log.gz 2024-04-26T20:27:02.318 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59016.log 2024-04-26T20:27:02.318 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42315.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42680.log.gz 2024-04-26T20:27:02.318 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42315.log.gz 2024-04-26T20:27:02.318 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58602.log 2024-04-26T20:27:02.318 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60945.log 2024-04-26T20:27:02.318 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59016.log.gz 2024-04-26T20:27:02.319 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.58602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58602.log.gz 2024-04-26T20:27:02.319 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55628.log 2024-04-26T20:27:02.319 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59716.log 2024-04-26T20:27:02.319 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55628.log: /var/log/ceph/ceph-client.admin.60945.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55628.log.gz 2024-04-26T20:27:02.319 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60945.log.gz 2024-04-26T20:27:02.319 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.59716.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59716.log.gz 2024-04-26T20:27:02.320 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T20:27:02.320 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T20:27:02.367 INFO:teuthology.orchestra.run.smithi134.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T20:27:02.370 INFO:teuthology.orchestra.run.smithi037.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T20:27:02.372 INFO:teuthology.orchestra.run.smithi134.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T20:27:02.388 INFO:teuthology.orchestra.run.smithi037.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T20:27:02.396 INFO:teuthology.orchestra.run.smithi134.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T20:27:02.404 INFO:teuthology.orchestra.run.smithi134.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T20:27:02.414 INFO:teuthology.orchestra.run.smithi037.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T20:27:02.421 INFO:teuthology.orchestra.run.smithi134.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T20:27:02.423 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-04-26T20:27:02.423 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m0.307s 2024-04-26T20:27:02.423 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m1.633s 2024-04-26T20:27:02.423 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.071s 2024-04-26T20:27:02.445 INFO:teuthology.orchestra.run.smithi037.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T20:27:02.504 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T20:27:03.155 INFO:teuthology.orchestra.run.smithi037.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T20:27:03.157 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-26T20:27:03.157 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m1.062s 2024-04-26T20:27:03.157 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m2.444s 2024-04-26T20:27:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.251s 2024-04-26T20:27:03.158 INFO:tasks.ceph:Archiving logs... 2024-04-26T20:27:03.158 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165/remote/smithi037/log 2024-04-26T20:27:03.159 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T20:27:03.757 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165/remote/smithi134/log 2024-04-26T20:27:03.758 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T20:27:04.002 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T20:27:04.011 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T20:27:04.011 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T20:27:04.014 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T20:27:04.098 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T20:27:04.106 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T20:27:04.107 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:27:04.109 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:27:04.122 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:27:04.124 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:27:04.126 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:27:04.127 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-26T20:27:04.127 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +1378us[+1385us] +/- 54ms 2024-04-26T20:27:04.127 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -1034us[-1026us] +/- 72ms 2024-04-26T20:27:04.127 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -975us[ -975us] +/- 84ms 2024-04-26T20:27:04.127 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:27:04.129 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:27:04.129 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-26T20:27:04.129 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +1366us[+1392us] +/- 54ms 2024-04-26T20:27:04.129 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 -1051us[-1025us] +/- 72ms 2024-04-26T20:27:04.129 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -1003us[ -978us] +/- 84ms 2024-04-26T20:27:04.129 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:27:04.130 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T20:27:04.139 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T20:27:04.140 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T20:27:04.182 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T20:27:04.256 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T20:27:04.318 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:27:04.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:27:04.347 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:27:04.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:27:04.374 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T20:27:04.384 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T20:27:04.407 INFO:teuthology.task.internal:Duration was 690.610707 seconds 2024-04-26T20:27:04.407 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T20:27:04.449 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T20:27:04.450 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T20:27:04.452 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T20:27:04.489 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:27:04.490 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:27:04.834 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T20:27:04.834 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T20:27:04.835 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T20:27:04.896 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-04-26T20:27:04.896 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T20:27:04.924 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T20:27:04.924 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T20:27:04.938 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T20:27:04.988 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T20:27:04.989 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T20:27:05.005 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T20:27:05.129 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T20:27:05.139 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T20:27:05.139 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T20:27:05.164 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T20:27:05.187 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T20:27:05.196 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:27:05.207 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:27:05.228 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-04-26T20:27:05.249 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-04-26T20:27:05.261 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:27:05.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:27:05.295 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:27:05.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:27:05.315 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T20:27:05.325 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T20:27:05.326 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165/remote/smithi037 2024-04-26T20:27:05.326 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T20:27:05.369 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165/remote/smithi134 2024-04-26T20:27:05.369 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T20:27:05.400 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T20:27:05.400 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T20:27:05.406 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T20:27:05.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T20:27:05.461 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T20:27:05.461 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T20:27:05.491 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T20:27:05.492 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T20:27:05.494 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T20:27:05.507 INFO:teuthology.orchestra.run.smithi037.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 20:27 /home/ubuntu/cephtest 2024-04-26T20:27:05.508 INFO:teuthology.orchestra.run.smithi134.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 20:27 /home/ubuntu/cephtest 2024-04-26T20:27:05.509 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T20:27:05.533 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T20:27:05.618 INFO:teuthology.nuke:Checking targets against current locks 2024-04-26T20:27:05.641 DEBUG:teuthology.nuke:shortname: smithi037 2024-04-26T20:27:05.641 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:27:05.665 DEBUG:teuthology.nuke:shortname: smithi134 2024-04-26T20:27:05.666 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:27:05.669 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:10:06.308296', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFJI9/a6xN71scHvRs+a5LoMYoCg+MPzsxjdSAWc0S4CYiqDJV8Blh4sl0X8rm8epzpjduzbDy8WjrR3AzVGZxs='} 2024-04-26T20:27:05.695 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:10:06.309815', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOiFV00u1Fzyoluyqeb6wqgjcP6WsSXZvL9D3fU5DC4maJJdoBJHMG53mJJygw64fB7I3ioVgzYq8DCD6mjgbgA='} 2024-04-26T20:27:05.726 INFO:teuthology.orchestra.console.smithi037:Power off 2024-04-26T20:27:05.727 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T20:27:05.752 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2024-04-26T20:27:05.752 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:27:05.770 INFO:teuthology.orchestra.console.smithi134:Power off 2024-04-26T20:27:05.770 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T20:27:05.781 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-04-26T20:27:05.794 DEBUG:teuthology.orchestra.console.smithi134:power off output: Chassis Power Control: Down/Off 2024-04-26T20:27:05.794 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:27:05.819 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-04-26T20:27:09.783 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:27:09.820 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:27:09.894 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-04-26T20:27:09.929 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-04-26T20:27:13.896 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:27:13.930 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:27:14.008 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2024-04-26T20:27:14.039 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-04-26T20:27:14.110 INFO:teuthology.orchestra.console.smithi037:Power off completed 2024-04-26T20:27:18.044 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:27:18.155 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is off 2024-04-26T20:27:18.256 INFO:teuthology.orchestra.console.smithi134:Power off completed 2024-04-26T20:27:18.361 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone}} duration: 690.61070728302 failure_reason: 'Test failure: test_clone_failure_status_in_progress_cancelled (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=e3e69736361a443b8dd71e3500ecf9d0 status: fail success: false 2024-04-26T20:27:18.362 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:27:18.446 INFO:teuthology.run:FAIL